Download specific files with FTP?

Question

I am trying to use FTP to download several hundred files within a directory. Is there a way to only download specific files with FTP?

Ideally, I would like to match specific files using a regular expression. Unfortunately, the directory is massive and it's too large to download the entire directory and then use find or grep locally to manipulate certain files. Is there a simply way to do what I am trying to do on Unix?

If not, I plan to just write a script in perl or python. Thanks in advance for the help.

Answer 1

You can use shell globbing on the remote system by making sure glob is on, then doing mget.

See http://www.proftpd.org/docs/howto/Globbing.html for more information.

I believe that's as far as you can get with FTP. If you can get ssh acess instead, there is much more that you can do, including grepping and modifying the files on the server.

Answer 2

Very often large file repositories will have an index.txt file somewhere about. If you download that it contains a list of all the files inside that folder. Quite useful for quick greps of package lists at least.

Answer 3

Mount the FTP server as a local directory with one of the several FUSE-based filesystems for FTP, then use your favorite tool (such as cp plus shell wildcards, or rsync) to match and copy the files you want to retrieve. For example:

mkdir example-somedir
curlftpfs ftp://ftp.example.com/somedir example-somedir
cd example-somedir
rsync -a --exclude 'boring.ext' --include='*.ext' --exclude='*' . /path/to/local/copy
fusermount -u example-somedir

Answer 4

If you want a GUI solution and you have the Konqueror browser, you can type ftp:// and the ftp address and browse the directory structure.

It sounds like you may want a command line solution, however.