2

I'm running a rsync command between two servers with the following command:

rsync -auv -P  --delete --delete-excluded  --stats --progress -e "ssh -i ~/.ssh/$ssh_pkey -p $port -l $user" $server_ip:$src/ $dest/

At some stage, an HDD failed on the source server, its mount point becoming inaccessible. This happened during the rsync command. Unfortunately, all the files on my destination folder got deleted while there was no --ignore-errors option in the rsync command.

Here are the logs:

rsync: read errors mapping "/data/TEST1/file.tmp": Input/output error (5)
rsync: send_files failed to open "/data/TEST1/file.tmp": Input/output error (5)
.............
rsync: opendir "/data/TEST1/Stranger Things" failed: Input/output error (5)
rsync: send_files failed to open "/data/TEST1/file2.tmp": Input/output error (5)
  3,511,822,850 100%    1.84MB/s    0:30:18 (xfr#36, ir-chk=1013/7161)
WARNING: TEST1/file.tmp failed verification -- update discarded (will try again).
deleting TEST1/file3.tmp
deleting  ...
deleting ...
deleting ...
....

After noticing the I/O error, the rsync command proceeds to delete all the remnant files!!

This is a very dangerous scenario. So far no harm was done, as I just had to re-mount the HDD, restart the rsync (which will take a few days), but next time I might not be as lucky.

What exactly did go wrong? How to prevent this?

1 Answer 1

2

This is expected behavior with the --delete argument. You can use --delete-after to wait for the transfer to finish before deleting unnecessary files.

man rsync

...
     --del                    an alias for --delete-during
           --delete                 delete extraneous files from dest dirs
           --delete-before          receiver deletes before xfer, not during
           --delete-during          receiver deletes during the transfer
           --delete-delay           find deletions during, delete after
           --delete-after           receiver deletes after transfer, not during
           --delete-excluded        also delete excluded files from dest dirs
           --ignore-missing-args    ignore missing source args without error
           --delete-missing-args    delete missing source args from destination
           --ignore-errors          delete even if there are I/O errors
...
2
  • I find this extremely weird that it's intended to be like this. It's very dangerous, and makes rsync not very usable in an automated way Commented Feb 16, 2022 at 0:17
  • 1
    @LaurentBesnard Most utilities that delete files are dangerous when used in the wrong way. This answer shows that it's possible to protect against that danger and how to do that, so it's not as if the fault is with rsync. Commented Feb 16, 2022 at 7:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.