rsync hangs on small files with no error messages

Question

I'm using rsync -rvhe ssh --partial-dir=path/to/partial/dir dir_to_copy remote_server:path/to/target/directory to attempt to copy a local directory to a remote server. I have also attempted this without using the --partial-dir arguement. The local copy of rsync is version 3.1.3 (included in WSL) and the remote version is 3.1.1 (included in the remote Ubuntu installation). I have already successfully completed this process for most of the files, which I did using compression (-z) and excluding the large files, and now I am attempting to complete the process by copying the large files as well.

However, rsync just hangs on certain files. This occurs both on 1) transfers and on 2) checking if a transfer has already completed. To further elaborate on this, for 1), I see the partial transfer appear in the /path/to/partial/dir (I've also seen the partial file in the target directory if I omit the --partial-dir command, as expected), and sometimes the transfer even completes in the partial directory before things stop moving. When 2) occurs, the transfer simply stops, even though the file is already present on the remote server.

Furthermore, rsync continues to run on my local machine and on the remote server, it just stops doing anything. As a result, it doesn't throw any errors for me to diagnose the issue. When I kill the local task using ^C, it returns rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(644) [sender=3.1.3], but it doesn't immediately return me to the terminal prompt--it hangs for a while after it says it received the SIGINT before printing [sender] io timeout after 60 seconds -- exiting and then returning me to the terminal prompt.

This sounds similar to what was described here: rsync keeps stopping on one directory with no errors , but I can't use their solution, which was to use cp, as I have over 10 GB to transfer and don't want to risk the copy getting interrupted and having to start from scratch.

Does anyone have any ideas how I can identify the root cause and fix this issue? Thanks!

Answer 1

You've missed the -t flag to capture times, so rsync has to fall back to checksums for verifying copies. This is extraordinarily slow and may well be the cause of the apparent slowdowns.

Try this instead,

rsync -rtvh dir_to_copy remote_server:path/to/target/directory

Rerun both your "small files only" copy and then the "all files" copy to see the effect.