Downloading files over ssh based on a list of filenames

Question

So here is what I have done already:

I have a file called abc.txt which contains list of files. I am using abc.txt to move those files to a folder, tar that folder and finally I download the tar to my local PC from my server, which is running GNU/Linux.

Here are the steps in list form:

1. abc.txt
2. abc.txt (listed files) -> folder
3. Folder -> folder.tar
4. folder.tar -> local PC.

If abc.txt contains 2 files, for example:

example1.css
example2.css

I need to download those files from abc.txt separately and directly to the local PC.

Since ftp or sftp need the file name to download it, how can I read that from abc.txt?

Answer 1

If the file abc.txt contains the list of filenames relative from /path/to/base:

ssh user@server tar c -T /path/to/abc.txt -C /path/to/base | (cd /tmp; tar xv)

This creates a tarball on-the-fly, without actually saving it anywhere, pipe it to the local shell, extract, effectively copying the listed files.

EXTRA TIPS

If the file abc.txt contains the list of absolute paths:

ssh user@server tar c -T /path/to/abc.txt | (cd /tmp; tar xv)

If the file abc.txt is on your local system, not the remote:

ssh user@server tar c -T- < /path/to/abc.txt | (cd /tmp; tar xv)

To use gzip compression (using default level 6):

ssh -C user@server tar c -T /path/to/abc.txt | (cd /tmp; tar xv)

To use gzip compression level 9:

ssh user@server 'tar c -T /path/to/abc.txt | gzip -9' | (cd /tmp; tar zxv)

Answer 2

Here is a simple script witch works with files containing spaces in the filename :

SAVEIFS=$IFS; IFS=$(echo -en "\n\b") ; for elt in `cat abc.txt` ; do scp $elt <YOUR_SFTP_CONNECTION> ; done ; IFS=$SAVEIFS

Answer 3

I think the hub of your problem is how to extract the correct files from your list for your subsequent two logic paths.

egrep 'example1.css|example2.css' abc.txt

will give you all lines that match the exceptions, and

egrep -v 'example1.css|example2.css' abc.txt

will give you all lines that don't match the exceptions