Loop over files in directory - save their name to a file and then change name

Question

As it is written in the title I want to write a program in bash that will do following:

• Loop over each file in a directory (there will be 100 files in each directory)

• Assign to it random number between 1 and 100 that was not assigned yet

• Save file name to another file (results.txt) in format X(random number that was assigned) file name

• Change the name of the file to the random number

I know how to loop over files but rest is a bit beyond my capabilities. I Would be very thankful for help in that matter :)

Answer 1

The main difficulty with this exercise is to generate a unique random number for each of the 100 files. The solution to that problem is to generate the numbers that we know we need (1 through to 100), and then scramble them (or scramble the list of filenames) before pairing a number with a filename. What you don't want to end up doing is to roll your 100-side dice, check if you've already rolled that number, and re-roll if you have, until you get a number you haven't rolled before. For a large number of files, that procedure might take a very long time. (Fun fact, we used this as an interview question some years ago).

Assuming no filename contains embedded newlines:

paste <( printf '%d\n' {1..100} ) <( printf '%s\n' dir/* | sort -R ) >result.txt

This would create two tab-delimited columns with the paste utility. The first column contains the integers 1 through to 100 in order. The second column contains the names of the 100 files in the directory dir (including the directory name). The list of names is sorted in a random order (the -R option to sort is non-standard, but commonly available).

You may also leave the filenames sorted and instead scramble the integers:

paste <( printf '%d\n' {1..100} | sort -R ) <( printf '%s\n' dir/* ) >result.txt

To change the names of the files, read the result.txt file:

while IFS= read -r stuff; do
    number=${stuff%%$'\t'*}   # the thing before the first tab
    pathname=${stuff#*$'\t'}  # the thing after the first tab
    mv -i -- "$pathname" "$(dirname -- "$pathname")/$number"
done <result.txt

Running this over all subdirectories in the current directory (think before running this, and always keep a backup of important data):

for dirpath in */; do
    paste <( printf '%d\n' {1..100} | sort -R ) <( printf '%s\n' "$dirpath"/* )
done >result.txt

while IFS= read -r stuff; do
    number=${stuff%%$'\t'*}
    pathname=${stuff#*$'\t'}
    mv -i -- "$pathname" "$(dirname -- "$pathname")/$number"
done <result.txt

or, more streamlined,

for dirpath in */; do
    paste <( printf '%d\n' {1..100} | sort -R ) <( printf '%s\n' "$dirpath"/* )
done |
tee result.txt |
while IFS= read -r stuff; do
    number=${stuff%%$'\t'*}
    pathname=${stuff#*$'\t'}
    mv -i -- "$pathname" "$(dirname -- "$pathname")/$number"
done

Answer 2

This answer uses the perl rename utility (aka prename or file-rename). NOT to be confused with the rename from util-linux, which has completely different command-line options and capabilities, or any other rename command.

One of the very nice things about the perl rename utility is that you can not only do relatively trivial sed-like transformations on filenames (e.g. rename 's/foo/bar/' *), you can use ANY algorithm implemented in perl to rename files. Each filename is held in perl's $_ special variable and will be renamed iff $_ is changed.

Which is what the following rename one-liner does:

$ rename -n '
  BEGIN{
    open(RESULTS,">","results.txt");
  };

  our $rnd = 0;
  our @used;

  # find a random number from 1..100 that hasnt been used yet.
  until (($rnd > 0) && (!defined($used[$rnd]))) {
    $rnd=int(rand(100)+1);
  };
  $used[$rnd]=1;

  print RESULTS "$rnd\t$_\n";
  $_ = $rnd' *

The -n option to rename makes this a dry-run, which shows what it would do if you let it.

Remove the -n (or replace it with -v for verbose output) to make it actually rename the files.

BTW, rename can take the list of filenames to be renamed as command-line arguments, or from STDIN, or both. Also worth noting is that it supports a -0 option for NUL-separated input (e.g. from find ... -print0).

NOTE: the results.txt file can't be reliably used to reverse the renaming if any of the original filenames contained \n newline characters. If you can't be certain of that, then use NULs to separate each record in results.txt instead of newlines. i.e. replace the second-last line of the rename script with:

  print RESULTS "$rnd\t$_\0";

---

FYI, to reverse:

$ rename -n '
  our %files;

  BEGIN{
    open(RESULTS,"<","results.txt");
    # local $/ = "\0"; # uncomment for NUL-separated input
    while(<RESULTS>){
      chomp;
      my ($n,$f) = split /\t/,$_,2;
      $files{$n} = $f;
    };
    close(RESULTS);
  };

  if (defined($files{$_})) { $_ = $files{$_} };
  ' [0-9]*

The ,2 in the split /\t/,$_,2 line limits the split to a maximum of two fields - even if there is more than one tab character in the input record...so tab characters in the filename won't break the script.

A naive reversal like sed -e 's/^/mv /' results.txt | sh would break on filenames containing spaces, tabs, etc or shell meta-characters.

---

FYI: I just read Kusalananda's answer, and feel that it's worth pointing out that this answer does just repeatedly roll d100 and check if we've already used that random number.

That's partly because there's little or no performance impact for only 100 random numbers, and partly because it's far less of a performance issue in perl....bash is slow, and loops in bash are especially slow. But mostly because it didn't occur to me until I read his answer :)

In theory, this loop could be endless (with a very bad sequence of random numbers) or take a very long time. In practice, both outcomes are extremely unlikely, although generating an unused number is likely to take longer for each iteration of the loop (but this is unlikely to even be noticeable by a human).

It is possible to pre-generate a randomised array as in K's answer, e.g. using the List::Util module's shuffle() function.

$ rename -n '
  use List::Util qw(shuffle);
  our $i;
  our @random;

  BEGIN{
    @random = shuffle 1..100;
    $i=0;
    open(RESULTS,">","results.txt");
  };

  print RESULTS "$random[$i]\t$_\n";
  $_ = $random[$i++]' *

Answer 3

With zsh:

n=0; for f in dir/*(noe['REPLY=$RANDOM']); do
  mv -i -- $f $f:h/$((++n)) &&
    print -r -- $f was renamed to $n
done > result.txt

Where the oe glob qualifier defines the order of the glob expansion based on the evaluation of the provided expression (which here returns a random number between 0 and 32767) which in effects gives you a shuffled like of files.

We then rename those files in that order to an increasing number, so it would work regardless of how many files there are in the directory.