I'm trying to synchronize literally thousands of files of various sizes and I would like to have a 1:1 copy of the files. That means that already present files should be checked for their integrity and if there's a wrong checksum, the file needs to be overwritten. A so-called delta transfer is only necessary at this point because of the partially failed transfer.
Apparently my mount is kinda unstable and it fails after 300-400GB of transfer using cp or rsync.
I did the following before this:
- I mounted the storage, and did
cp -r src dest, it failed after like 300GB because the mount dropped and it errored out (don't have the error anymore apparently) - I mounted the storage again and did
rsync -aP src dest, it failed after like 400GB withrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]because the mount failed again. Considering the file size it probably overwrite most of the files. - I checked my kernel log and found nothing (
sudo dmesg)
I found a reconnect flag for my mount, but it would not be instant.
- There's a rsync flag named
-cwhich calculates the checksums, but does it do a so-called delta transfer too or do I need to add more flags?
How could I best fix this problem at hand?
UPDATE 1
Correct me if I'm wrong but I think the issue at hand was that the storage had the owners and groups of different users and groups than in rsync. To elaborate: cp -r copied the files and changed their ownership and group ownership to the user copying, whereas rsync seems to copy the file 1:1 with the same user and group ownership... That's probably why the transfer was overwriting old files...