Return to Question

What I want:

I'm trying to back up many files to S3 each day. I want to compress these into archives daily so that each individual backup doesn't cost as much. Therefore, I want a backup system whereby I can create differential archives from the previous day to be backed up to S3.

My current reasoning is to create an initial backup, and then base backups per day that are differential wrt that initial backup.

Example:

Say I have a directory named dir containing a file, a.txt:

dir:
  a.txt

Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:

dir:
  a.txt
  b.txt

Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.

EDIT:

Halfway there!

rsync -n -rpgov backup/ backup-copy/

Please note the -n means this does a dry-run,, so you can happily test this without wrecking your own local directories with file changes.

This outputs the difference between two local folders. However, I cannot for the life of me figure out how to get this output, through rsync's options, to a new directory.

I've tried using rsync's --compare-dest flag:

rsync -n -rpgov --compare-dest=backup/ backup-copy/

but this seems to output different files to the above command. Any help to push this over the edge?

What I want:

I'm trying to back up many files to S3 each day. I want to compress these into archives daily so that each individual backup doesn't cost as much. Therefore, I want a backup system whereby I can create differential archives from the previous day to be backed up to S3.

My current reasoning is to create an initial backup, and then base backups per day that are differential wrt that initial backup.

Example:

Say I have a directory named dir containing a file, a.txt:

dir:
  a.txt

Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:

dir:
  a.txt
  b.txt

Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.

EDIT:

Halfway there!

rsync -n -rpgov backup/ backup-copy/

Please note the -n means this does a dry-run,, so you can happily test this without wrecking your own local directories with file changes.

This outputs the difference between two local folders. However, I cannot for the life of me figure out how to get this output, through rsync's options, to a new directory.

I've tried using rsync's --compare-dest flag:

rsync -n -rpgov --compare-dest=backup/ backup-copy/

but this seems to output different files to the above command.How to push this over the edge?

I'm trying to back up many files to S3 each day. I want to compress these into archives daily so that each individual backup doesn't cost as much. Therefore, I want a backup system whereby I can create differential archives from the previous day to be backed up to S3.

My current reasoning is to create an initial backup, and then base backups per day that are differential wrt that initial backup.

Say I have a directory named dir containing a file, a.txt:

dir:
  a.txt

Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:

dir:
  a.txt
  b.txt

Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.

EDIT: What I basically want is a system for backups which creates separate archives of the differences since the last specified backup.

Using the example above, I'd want two archives; the first containing only a.txt and the second containing only b.txt. I want to do this locally first.

What I want:

I'm trying to back up many files to S3 each day. I want to compress these into archives daily so that each individual backup doesn't cost as much. Therefore, I want a backup system whereby I can create differential archives from the previous day to be backed up to S3.

My current reasoning is to create an initial backup, and then base backups per day that are differential wrt that initial backup.

Example:

Say I have a directory named dir containing a file, a.txt:

dir:
  a.txt

Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:

dir:
  a.txt
  b.txt

Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.

EDIT:

Halfway there!

rsync -n -rpgov backup/ backup-copy/

Please note the -n means this does a dry-run,, so you can happily test this without wrecking your own local directories with file changes.

This outputs the difference between two local folders. However, I cannot for the life of me figure out how to get this output, through rsync's options, to a new directory.

I've tried using rsync's --compare-dest flag:

rsync -n -rpgov --compare-dest=backup/ backup-copy/

but this seems to output different files to the above command. Any help to push this over the edge?

Say I have a directory named dir containing a file, a.txt:

dir:
  a.txt

Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:

dir:
  a.txt
  b.txt

Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.

EDIT: What I basically want is a system for backups which creates separate archives of the differences since the last specified backup.

Using the example above, I'd want two archives; the first containing only a.txt and the second containing only b.txt. I want to do this locally first.

I'm trying to back up many files to S3 each day. I want to compress these into archives daily so that each individual backup doesn't cost as much. Therefore, I want a backup system whereby I can create differential archives from the previous day to be backed up to S3.

My current reasoning is to create an initial backup, and then base backups per day that are differential wrt that initial backup.

Say I have a directory named dir containing a file, a.txt:

dir:
  a.txt

Then let's say I run rsync -r dir backup.1, or some other backup program, such as, but not limited to, rdiff-backup. Great stuff, I have a backup located at backup.1.

Now, I add one file, b.txt, so that folder's contents is as follows:

dir:
  a.txt
  b.txt

Is there an rsync, or another backup utility, command I can run that will give me a separate directory, backup.2, that contains only the file b.txt?

If I modify the permissions of a.txt, will that file also be included?

What I've tried:

I've tried rsync and rdiff-backup, with a range of flags, but I cannot get anywhere. I've been doing it for hours now, and I cannot figure it out.

EDIT: What I basically want is a system for backups which creates separate archives of the differences since the last specified backup.

Using the example above, I'd want two archives; the first containing only a.txt and the second containing only b.txt. I want to do this locally first.