0

I've recently been experimenting with using rsync as a means of version control/backup for certain files on my system.

My goal is to essentially have rsync set to monitor some specific files, and if/when those files change for rysnc to then make a copy of said files to another destination, as a separate file (rather than overwriting the initial backed-up file).

So currently, what I do have is this:

rsync ~/Database/original.sqlite ~/Backup/backup`date +'.%F_%H-%M-%S'`.sqlite

I have this running as an Agent in launchd on a 1 minute timer. So with how it is now, essentially rsync will just keep making new copies of this file every minute no matter what.

If I had a means for rsync to check the latest version in the Backup folder, compare to original file in Database folder, then if the files are different make a new backup version, that would be perfection.

Am I really stretching the limits of rsync with what I'm trying to do here?

Thank you in advance for any suggestions.

1
  • What neither you nor (currently) the answers address is that the database file copy will be corrupt if anything is writing to it during the copy process. Copying isn't instantaneous Commented Oct 5, 2022 at 12:09

2 Answers 2

0

If this is always running, the simplest solution to me would be just to check if the file has been edited in the last minute, and if so, do the backup.

#!/bin/bash

# get current time
now="$(date +'%s')"

# get modification time of fime
modified="$(stat -c '%Y' ~/Database/original.sqlite)"

# check if modified in the last minute
if (( modified > (now - 60) )) ; then
    rsync ~/Database/original.sqlite ~/Backup/backup`date +'.%F_%H-%M-%S'`.sqlite
fi

If you want it to be really robust, you could either write the name of the last backup, or its timestamp, to a file, and then check against that. Not sure if you're trying to keep it simpler than that, however.

I suppose another option would be to have the script itself be in charge of firing every minute and have launchd just run it once in the background at boot. Then the script could keep track the last modification time itself:

#!/bin/bash

# initialize last backup at 0
lastbackup=0
while true ; do

    # get modification time
    modified="$(stat -c '%Y' ~/Database/original.sqlite)"

    # check to see if it's been changed
    if (( modified > lastbackup )) ; then

        # do backup
        rsync ~/Database/original.sqlite ~/Backup/backup`date +'.%F_%H-%M-%S'`.sqlite

        # set the variable to its current modification time
        lastbackup="$modified"

    fi
done
5
  • Ran this script but getting error: usage: stat [-FlLnqrsx] [-f format] [-t timefmt] [file ...] stat: illegal option -- c @frabjous Commented Oct 8, 2022 at 16:49
  • What version of stat are you using? I was using GNU stat. I don't know what options are available for other versions. Commented Oct 8, 2022 at 17:12
  • It works with busybox stat too. All the command does is get the modification time of the file as a unix timestamp. I would think that every implementation of stat has some way to do that. Commented Oct 8, 2022 at 20:44
  • I'm using OSX 12.2.1 Commented Oct 11, 2022 at 21:59
  • I don't have access to a mac to test, but I gather from this answer here that you might try to change stat -c '%Y' to stat -f %m. Commented Oct 11, 2022 at 23:41
0

You can probably use:

rsync -au ~/Database/original.sqlite ~/Backup/

The -u option means

--update, -u skip files that are newer on the receiver

So, rsync will only overwrite files in the destination folder when it's newer on the source.

Then, you can monitor changes of your specific files in the destination folder, and make copies if necessary.

-- Update --

To monitor the change of the file, you can do something along the lines of the following script (adapted from the linked question). Note that this requires inotify-tools (sudo apt install inotify-tools in Ubuntu)

#! /bin/bash

f=~/Backup/original.sqlite
touch $f

inotifywait -m -e modify "$f" --format "%e" | while read -r event; do
    if [ "$event" == "MODIFY" ]; then
        prev="$curr"
        curr=$(<"$f")
        [ "$curr" == "$prev" ] || echo "$f changed: do something"
    fi
done

If you run the script, it will monitor change. The script just does echo "$f changed: do something" as an example, but you can adapt it to do anything you like. e.g. cp $f destination_file.

3
  • How would I make copies when a change is noticed? Commented Oct 6, 2022 at 14:06
  • @SkRevo Please see the update. Commented Oct 6, 2022 at 21:49
  • Got it, thank you, that's very clever. I went ahead with the other solution above as it was a bit more all encompassing but this would definitely work too. Thanks again for your help! Commented Oct 7, 2022 at 22:05

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.