Find and replace a string in a batch of files but do not create any filename conflicts

Question

I have a directory tree with a few thousand files collected over the years with random and useless file names.

I want to clean them up, keep them relevant, remove certain words but also not create any duplicate filenames.

For example

WONDERBROS - PAWG Remy LaCroix Gets BREAKFAST! (pwg11717) - SOMENAME.mp4

I want that one to be renamed something like

WONDERBROS.PAWG.Remy.LaCroix.Gets.BREAKFAST!.(pwg11717).mp4

So a few points are

• replace all spaces with .
• remove SOMENAME whenever it occurs
• Check that this filename does not already exist before renaming and if it does, append a number and keep trying until it is a unique filename so that nothing is overwritten

I think that this can be done with a shell script but if anyone has a tool to recommend I would like to try that too.

I tried something like this and only the first line seems to work.

# find and replace spaces with .
find /home/matt/rename_test_tmp/ -depth -name "* *" -execdir rename 's/ /./g' "{
# find and replace somename with .
find /home/matt/rename_test_tmp/ -depth -name "somename" -execdir rename 's/ 
# find and replace SOMENAME
find /home/matt/rename_test_tmp/ -depth -name "SOMENAME" -execdir rename 's/ 
# find and replace Somename
find /home/matt/rename_test_tmp/ -depth -name "Somename" -execdir rename 's/ 

I think my problem is that I still do not understand reg ex?

Answer 1

Here is a basic structure. If something goes wrong, MAXTRYS stops the renaming. Chose a reasonable value.

#!/bin/bash
#
# Usage: 
#       numberedMove xy-file.txt TARGETDIR 
#
file="$1"
targetDir="$2"
MAXTRYS=666
#
# @TODO
# add tests for permissions, maybe handle symlinks etc.
#
test -f "$file" || {
    echo No such file "$file" or not an ordinary file
    exit 1;
}

test -d "$targetDir" || {
    echo No such directory "$targetDir"
    exit 2;
}

#
# append NUM+1 to filename, to create unique name
#
numbered () {
  fname="$1"
  num=$2
  if [[ $num -gt $MAXTRYS ]]
  then
    echo " giving up - max trys: $MAXTRYS "
    exit 3
  fi
  # echo "mv $fname → $targetDir/$fname$num"
  test -f "$targetDir/$fname$num" && numbered "$fname" $((num+1)) || mv "$file" "$targetDir/$fname$num"
}

#
# remove all SOMENAME and replace every blank with a dot
#
filtername () {
   fname="$1"
   fname=${fname//SOMENAME/}
   fname=${fname// /.}
   echo "$fname"
}
#
# filter filename, if unique: move, else move with number suffix
#
if [[ -e "$file" ]]
then
    newname=$(filtername "$file")
    if [[ ! -f "$targetDir/$newname" ]]
    then
        # echo "mv $file → $targetDir/$newname"
        mv "$file" "$targetDir/$newname"
    else
        # echo "mv $file → $targetDir/$newname NUMBERED"
        numbered "$newname" 1
    fi
fi

In filtername you may add more rules and change the existing ones.

If this was for my own needs, I would replace blanks with '-', not '.', since the dot is a, fair enough, weak indicator (a.tar.bz2) for meaning.

And I would collapse multiple delimiter to one, so instead of:

    mv "ab cd - SOMENAME..mp3" → "./B/ab.cd.-...mp3" 
    mv "ab cd - SOMENAME..mp3" → "./B/ab-cd-.mp3" 

And I would keep the file extension, and put the number in front of the last extension:

    mv "ab cd- SOMENAME..mp3"  → "./B/ab-cd-0.mp3" 
    mv "ab cd - SOMENAME..mp3" → "./B/ab-cd-1.mp3" 
    mv "ab cd -SOMENAME..mp3"  → "./B/ab-cd-2.mp3" 

since many programs interpret the extension.

I only tested the program with about 10 files, so make a backup before using, and check, maybe by total file size, and file count, whether the operation succeeded.

My testing:

for f in a* ; do ./numberedMove.sh "$f" ./B ; done

Note, that when placing the program in the CWD, it might get moved itself.

Answer 2

If you're using the perl rename utility, try something like this:

find . -type f -execdir rename -n 's/\s*-\s*/./g; s/\s+/./g; s/somename//ig' {} +

The \s*-\s* matches a - surrounded by zero-or-more whitespace characters (i.e. a - with optional whitespace). \s+ matches one-or-more whitespace characters.

You can add as many s/search/replace/ commands in the rename script as you like. Note, though, that you need to be careful about the order of execution - e.g. if you want to change "foo" to "bar" and "food" to "drink", then you need s/food/drink/ before s/foo/bar/ because the latter will change food to bard.

The command above is applied to ALL filenames. If some of the s/search/replace/ commands do not apply to a particular filename, it's not an error - that particular command is not applied (but others still are). Even so, you may need to run multiple find commands using the -name or -iname predicate if you want some renames to only be applied to certain file names.

You can also use alternation if you want to change multiple words to the same replacement. e.g.

s/(somename|someothername|thisname|thatname)//ig

that changes all of them to the empty string (i.e. removes them). The /i modifier makes it case-insensitive.

The -n option from rename is a dry-run option - it will only show what would be renamed, without actually renaming anything. Remove the -n from the rename command when you've verified that it does what you want, and nothing that you don't want. If you want verbose output while renaming, replace the -n with -v.

from man rename:

-v, -verbose

Verbose: print names of files successfully renamed.

-n, -nono

No action: print names of files to be renamed, but don't rename.

---

Note: The perl rename utility will not rename a file if the new filename already exists. It has no built-in ability to add numbers to filenames in case of collision, but one of the best features of this rename is that you're not restricted to just simple operations like s/search/replace/ - you can run any perl code inside a rename script and it will rename the source filename to whatever $_ is changed to (operators like s/// or y// implicitly modify $_ if an explicit variable name is not used. If you're new to perl, there's a pretty good explanation of what $_ is at https://perlmaven.com/the-default-variable-of-perl).

That allows something like this (untested, but might work)

find . -type f -execdir rename -n '
    s/\s*-\s*/./g; 
    s/\s+/./g;
    s/somename//ig;

    if (-f $_) {
      my $num='001';
      while (-f "$_.$num") {
        $num=sprintf('%03i',++$num);
      };
      $_ = "$_.$num";
    }' {} +

That should add a zero-padded 3-digit-wide number (i.e. from 001 to 999) to the filename if it already exists. Change %03i to %02i in the sprintf if you want only two digits (01 to 99). Or, if you don't expect any numbering of duplicate files to exceed 9, change it to just my $num=1, and then just $num++ inside the while loop with no sprintf().

with a little more work (by splitting the filename into "basename" and "extension" portions) the numbering could be inserted before the extension, rather than just appended to the end.

---

You can find out which version of rename you have with the -V option. e.g. on my Debian system rename is the perl rename while rename.ul is the util-linux rename:

$ rename -V
/usr/bin/rename using File::Rename version 0.20

$ rename.ul -V
rename.ul from util-linux 2.31.1

Answer 3

find . -maxdepth 1 -type f > orginal_file

find . -maxdepth 1 -type f -exec sed -r "s/\s+//g"| sed "s/SOMENAME//g" >> /var/tmp/modified file

paste orginal_file /var/tmp/modified file >> combined_orginal_modified_column_wise

for i in `cat /var/tmp/modified file`
do
if [[ -f $i ]]
then
echo "file exsist"
else
awk -v i="$i" '/i/{print "mv" " " $1 " " $2}'   combined_orginal_modified_column_wise
fi
done

for i in `cat /var/tmp/modified file`
do
if [[ -f $i ]]
then
for j in {1..10}
do
elif [[ -f $i$j ]]
then
echo "$i$j exsists"
fi
done
else
awk -v i="$i" -v j="$j" '{print "mv" " " $1 " " ij}'  combined_orginal_modified_column_wise
fi


done