move files of certain type and rename ones with the same name

Question

Here is what I am trying to achieve:

I converted pst files using readpst library into separate .eml files. After converting each of the pst files, I got a directory with multiple sub-directories nested, and each of those directories and sub-directories have those .eml files numbered from 1..n.

The problem is I need to have them all in one directory, however when I try to move them like:

find . -name '*.zip' -exec mv --target-directory='/path/to/outputdir' '{}' +

it finds those named the same and ends with:

mv: will not overwrite just created 

Does anyone have an idea how to move files into one directory and rename so they don't clash?

Answer 1

The GNU implementation of cp has a --backup=[CONTROL] option, that creates differently named files if the destination file already exists. The format is a little ugly, since it uses tilde symbols ('~') to delimit the backup extension.

A useful CONTROL argument in your case could be numbered:

find . -name '*.zip' -exec cp --backup=numbered '{}' '/path/to/outputdir' \;

Note that since I don't know of any such option for mv, if you take this approach, you'll later have to delete the original files in a separate step.

One issue I see with this solution is that it will be indistinguishable which file came from which directory. EDIT: Although some people find this behavior appropriate, and is the default behavior of browsers for instance, when you download same-named files into the same directory multiple times. Although the filename extension for these "backups" is different in that case, it is also numbered.

Instead of the above, you could also embed the path into every filename, for instance by changing slash delimiters to underscores. Although the results are more informative, you might get really long and ugly file names and there is a lot of room for errors:

find . -name '*.zip' | while read filePath; do cp "$filePath" '/path/to/outputdir'/"$(echo "$filePath" | sed -e 's#^\\.##' -e 's#/#_#g')"; done

Also, files that actually had underscores in them will look weird and it may be ambiguous if an underscore separated component of its filename was previously a pathname component or not.

Note that the slash character between '/path/to/outputdir' and the command starting with "$(echo "$filePath" |... is extremely important!

The first sed expression is there to avoid hidden files to be created if the top of the search tree specified to find started with a dot character. The second is just the substitution of path separator slashes with underscores.

The solution is also not very scalable, since you might have to make a lot of changes if you simply want to modify the find query, to avoid drastic results. For instance, if the find query also includes directories, cp will not handle those lines. Luckily, in this case the copy simply fails and you even get a message to your error stream. Overwriting stuff on your filesystem is not a joke!