I'm running an rsynca rsync command between two servers with the following command:
rsync -auv -P --delete --delete-excluded --stats --progress -e "ssh -i ~/.ssh/$ssh_pkey -p $port -l $user" $server_ip:$src/ $dest/
rsync -auv -P --delete --delete-excluded --stats --progress -e "ssh -i ~/.ssh/$ssh_pkey -p $port -l $user" $server_ip:$src/ $dest/
At some stage, aan HDD failed on the source server, its mount point becoming inaccessible. This happened during the rsyncrsync command. Unfortunately, all the files on my destination folder got deleted while there was no --ignore-errors option in the rsyncrsync command.
Here are the logs:
rsync: read errors mapping "/data/TEST1/file.tmp": Input/output error (5)
rsync: send_files failed to open "/data/TEST1/file.tmp": Input/output error (5)
.............
rsync: opendir "/data/TEST1/Stranger Things" failed: Input/output error (5)
rsync: send_files failed to open "/data/TEST1/file2.tmp": Input/output error (5)
3,511,822,850 100% 1.84MB/s 0:30:18 (xfr#36, ir-chk=1013/7161)
WARNING: TEST1/file.tmp failed verification -- update discarded (will try again).
deleting TEST1/file3.tmp
deleting ...
deleting ...
deleting ...
....
After noticing the I/O error, the rsync command proceeds to delete all the remnant files!!
This is a very dangerous scenario. So far no harm was done, as I just had to re-mount the HDD, restart the rsync (which will take a few days), but next time I might not be as lucky.
What exactly did go wrong? How to prevent this?
