Skip to main content
1 of 6

Exclude from rsync log file or output non-updated files

For rsyncs that involve a large number of potentially transferred files, I mean to have a "clean" log, with only what was transferred. I worked with stdout, since: 1) I did not know how to configure the log output (if that is even possible). I had yet another reason forcing me to work with stdout, with something like this, since: 2) I also want this to work with --dry-run. And perhaps yet another reason: 3) I mean to write scripts useful for transferring Linux-Linux, and Linux-msys2 (and viceversa) at least. Here, options for permissions (-p), owner (-o) and group (-g) may complicate things.

I combined command

rsync --recursive --perms --group --owner --times --one-file-system --sparse --stats --progress --update --out-format='%t %p %i %n %M %l' --delete --dry-run --include="*" --exclude="*" --log-file=rsync.log ./ ../trg

with

... | grep -E -v ' \.[fdLDS][\.c][\.s][\.t][\.p][\.o][\.g][\.u][\.a][\.x]'

following the description here. The first dot matches any item that is not being updated. The second character matches any item type. The remaining characters match any condition on the attributes (change/no change) Flag -v reverts matching. This presumably matches any item that is being updated, and only that. Items not updated, but with changed timestamp, would not be matched.

I finally combine this with tee, as in

rsync --recursive --perms --group --owner --times --one-file-system --sparse --stats --progress --update --out-format='%t %p %i %n %M %l' --delete --dry-run --include="*" --exclude="*" --log-file=rsync.log ./ ../trg | grep -E -v ' \.[fdLDS][\.c][\.s][\.t][\.p][\.o][\.g][\.u][\.a][\.x]' | tee rsync.2.log

which produces the rsync log file rsync.log, and my "custom" log file rsync.2.log.

On one hand, I wonder if there are cases I am missing, where this approach might fail (by "excessive" reporting or filtering files).

On the other hand, I wonder if I can handle this via rsync itself, and even better, to the rsync log file (not only its output), rather than using the brute force grep approach. Note that using --itemize-changes I get in stdout some files with e.g. status .f...p....., which should be filtered (as they are not updated, first character is a dot).

Related:

  1. how to make rsync to log deleted files
  2. How to make rsync display only files being copied, not all the folders which are scanned?
  3. Logging only transferred files with rsync
  4. https://superuser.com/questions/1002074/linux-command-line-to-create-a-log-file-for-rsync
  5. https://serverfault.com/questions/401210/rsync-report-only-uploaded-files