rsync - exclude all directories and files except a few

Question

I have to move some files/dirs from a directory that was shared by several people to an other computer. The elements that I have to move are some dirs, some subdirs, some specific files in subdirs, etc... It's a little mess ^^

I'm trying to apply the method in https://serverfault.com/a/788007/154366 but it fails somehow.

Here's my test setup (I tried to make it representative enough to be able to understand most of the intricacies of rsync --filter=merge so that I can apply it to my real problem):

#!/bin/bash
mkdir -p /tmp/test/{a/b,b/{a,b},c,.d}
touch /tmp/test/{f.txt,.f.txt,{a/b,b,b/{a,b},c,.d}/f.txt,b/.f.txt}
tree -a /tmp/test

/tmp/test
├── a
│   └── b
│       └── f.txt
├── b
│   ├── a
│   │   └── f.txt
│   ├── b
│   │   └── f.txt
│   ├── .f.txt
│   └── f.txt
├── c
│   └── f.txt
├── .d
│   └── f.txt
├── .f.txt
└── f.txt

Now I would like to use a single rsync -av command to copy the following items from /tmp/test to /tmp/filt:

• the complete ./a directory
• the complete ./b/b directory
• all the files & dotfiles direct children of the ./b directory
• all the files (not dotfiles) direct children of the ./ directory

My expected "copies" are marked with => here:

/tmp/test/           =>  /tmp/filt/
/tmp/test/a/         =>  /tmp/filt/a/
/tmp/test/a/b/       =>  /tmp/filt/a/b/
/tmp/test/a/b/f.txt  =>  /tmp/filt/a/b/f.txt
/tmp/test/b/         =>  /tmp/filt/b/
/tmp/test/b/a/
/tmp/test/b/a/f.txt
/tmp/test/b/b/       =>  /tmp/filt/b/b/
/tmp/test/b/b/f.txt  =>  /tmp/filt/b/b/f.txt
/tmp/test/b/f.txt    =>  /tmp/filt/b/f.txt
/tmp/test/b/.f.txt   =>  /tmp/filt/b/.f.txt
/tmp/test/c/
/tmp/test/c/f.txt
/tmp/test/.d/
/tmp/test/.d/f.txt
/tmp/test/f.txt      =>  /tmp/filt/f.txt
/tmp/test/.f.txt

So I wrote the following filter.txt file in order to use it with rsync's --filter=merge:

+ /[!.]*
+ /a/
+ /a/**
+ /b/
+ /b/b/
+ /b/b/**
- /b/*/
+ /b/*
- /**

But when I run the command:

rsync -av --dry-run --filter='merge filter.txt' /tmp/test /tmp/filt/

I get:

sending incremental file list
test/

sent 74 bytes  received 20 bytes  188.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

^{note: of course, the /tmp/filt directory doesn't exist, so the lack of output isn't related to it.}

What am I doing wrong?

Answer 1

You need to be careful with the difference between a source directory ending with / or not. The example answer you refer to has an ending slash. If you do the same it should work:

$ rsync -avn --filter='merge filter.txt' /tmp/test/ /tmp/filt/
created directory /tmp/filt
./
f.txt
a/
a/b/
a/b/f.txt
b/
b/.f.txt
b/f.txt
b/b/
b/b/f.txt
c/
sent 323 bytes  received 86 bytes  818.00 bytes/sec

The filter patterns are not working as they are compared with files like test/a/ instead of /a/.

To avoid copying directory c, you could remove the first and last line of the filter rules and add, for example, at the end 3 rules to exclude top-level dot files and directories, but include other files.

+ /a/
+ /a/**
+ /b/
+ /b/b/
+ /b/b/**
- /b/*/
+ /b/*
- /.*
- /*/
+ /*

The + /b/b/ and + /b/b/** can be collapsed into + /b/b/***.