Stabilize SSH During Rsync-Daemon Connection

Question

I am on Ubuntu 20.04 and I use the bash script / rsync below to connect to rsync-daemon on my remote server over ssh.

The downside is that using rsync over ssh also creates server load, therefore as to overcome the problem of server load, I added an ionice value as shown below. Load is no longer a problem.

My issue is that the ionice value seem to cause an unstable ssh connection and that in turn causes my login-notifier script to send a million emails because ssh keeps dropping and re-connecting continuously.

Might someone see a better solution to the load problem and/or know how to keep the ssh connection stable?

#!/bin/bash
while [ 1 ]
do
     rsync -avzxP --delete --checksum --append-verify --timeout=180 --bwlimit=48 --rsync-path="sudo ionice -c 3 rsync" --log-file=/var/log/rsync.log --exclude 'var-logs' --password-file=/etc/rsyncd.passwd -e "ssh -l backups" XXX.XX.XXX.XX::backup-data /media/username/WebMade/Server-Backups/Prod/today/
    if [ "$?" = "0" ] ; then
        echo "rsync completed normally"
        exit
    else
        echo "Rsync failure. Backing off and retrying..."
        sleep 10
    fi
done
#EOF

Answer 1

With this rsync command you're pulling files from the remote computer to your local computer.

The -z option asks the remote computer's rsync daemon to compress the file data that it streams through the network to your local rsync (and your local rsync decompresses the data before writing it to files). The ionice specified in the --rsync-path option reduces the disk i/o priority of the rsync process on the remote computer.

A process that's reading files from disk on a computer doesn't usually add a lot of load to the computer unless the disk is slow, so the ionice is not likely to help reduce load on the remote computer. With data compression, the process that's performing the compression often adds a lot of cpu load to the computer, while the process that's decompressing the data adds very little cpu load. The --bwlimit option can be useful when the file transfer consumes all the bandwidth of the network between the two computers, but a clogged network will not usually display itself as extra load in either computer.

In my experience, asking rsync to compress the data it transfers through the network (the -z option) is not very useful when the network connection between the computers works at the high speeds in a LAN or through most modern "high speed" Internet links to homes. You gain little or no reduction in the time it takes to transfer the files, at the cost of higher cpu consumption on the sending side.

Based on this, my advice is to remove the --bwlimit and --rsync-path and -z options. The last was most likely causing the server load you've been trying to solve.