Move files to directory according to the content pattern matching

Question

I would like to move files in existing directory files containing specific content to an existing or new directory and subdirectory by writing a script called fruit in ~/bin that moves them to dir/subdir.

For example, I have many regular files in existing directory files with name file1 file2 file3 .... file100.

ls:

file1
file2 
file3 
file4 
... 
file100

The contents of the files are:

cat file1

apple 1
789098

cat file2

orange 2
389342

cat file3

pear 1
678034

cat file4

grapes 3
123432

cat file5

apple 3
342534

cat file6

apple 3
234298

I would like to move files that have the same first line field1 content to a new directory with the same name as field1 while keeping the file name unchanged.

• that is file1, file5, file6 go to apple
• file2 goes to orange
• file3 goes to pear
• and so on

ls:

apple pear grapes orange etc ...

./apple:
file1 file5 file6

./pear:
file3

/orange:
file2

And then I would like to create a new subdirectory and move files that have the same first line field2 content to that subdirectory.

• under directoryapple, file1 will go to a subdirectory 1, file5 will go to a subdirectory 3, file6 will go to a subdirectory 3
• under directory orange, file2 will go to a subdirectory 2
• under directory pear, file3 will go to a subdirectory 1
• and so on

After sorting and moving, the files should be sorted to something like below:

ls:

apple pear grapes orange etc ...

./apple:
1 3

./apple/1:
file1

./apple/3:
file5 file6

./orange:
2

./orange/2:
file2

./pear
1

./pear/1
file3

How can I loop through all the files to move them to the suitable directory and subdirectory accordingly in shell with vi editor?

Answer 1

$ find . -name 'file*' 
./file6
./file1
./file5
./file2
./file3
./file4

$ perl -lane '
    close(ARGV);
    mkdir $F[0] unless -e $F[0];
    mkdir "$F[0]/$F[1]" unless -e "$F[0]/$F[1]";
    rename $ARGV, "$F[0]/$F[1]/$ARGV" if (-d "$F[0]/$F[1]");
  ' file*

$ find . -name 'file*' 
./pear/1/file3
./grapes/3/file4
./orange/2/file2
./apple/1/file1
./apple/3/file5
./apple/3/file6

file{1..6} are your sample files. The perl script opens each file in turn, reads in the first line and splits it into the @F array (via perl's -a command-line option). It then closes the file (which has the side-effect of resetting the line-counter, $.), creates the directories if they don't already exist, and moves the file into the directory if it's actually a directory (if it already existed, there's a chance it may be a regular file or symlink or something instead of a directory).

Files that don't have at least one line will be ignored. Files that differ from the expected format (i.e. first line contains two fields, separated by any kind of whitespace, with base directory name and subdir name) will cause undefined (possibly bizarre, possibly disastrous) results.

The two find commands are there to show the location of the files before and after running the perl one-liner. It's a bare minimum script and doesn't produce any output. It doesn't do anywhere near enough error checking or data validation, either.

---

Alternate version, as a standalone script. The only real reason for writing it is to address
Stéphane's comment about perl's -T option (which, in most cases, will not be a problem...but people do do pathologically crazy and even malicious things with filenames so caution/paranoia is not misplaced):

$ cat sort-move.pl 
#!/usr/bin/perl

use strict;
use File::Path qw(make_path);

while(<<>>) {
  my($dir,$subdir) = split;
  close(ARGV);
  make_path("$dir/$subdir");
  rename $ARGV, "$dir/$subdir/$ARGV" if (-d "$dir/$subdir");
}

Run it as, e.g., ./sort-move.pl file*. Aside from directory creation errors now being a fatal condition, results will be exactly the same as the one-liner version.

It does no extra error checking or data validation - actually, it does less (it relies on the make_path() function in the core perl File::Path module to create the directories - make_path() works much like mkdir -p). In other words, bad data can still make it do bad things, so don't feed it bad data.

The script will, however, now exit immediately with an error message if make_path fails because "$dir/$subdir" already exists and is not a directory (or for any other reason that causes an error, e.g. the filesystem being out of space or inodes). For example, if I ran mkdir apple; touch apple/1 before running this script, the error message would be mkdir apple/1: File exists at ./sort-move.pl line 9., no directories would be created by the script, and no files would be moved. I know this because I did exactly that to test it.

A complete script would handle error conditions gracefully. A complete script would also have a -n or --dry-run option, to only show what it would do without actually doing it. This is not a complete script, it is a minimal working example of one way to do what you want.

Answer 2

With the zsh shell (which has a vi line-editing mode like most shells as per your requirement, though I fail to see how that's relevant):

typeset -A files=()

for file (file*(N.L+3))
  read -r dir subdir ignore < $file &&
    [[    $dir != (|.|..|*/*) ]] &&
    [[ $subdir != (|.|..|*/*) ]] &&
    files[$dir/$subdir]+=$file$'\0'

if (($#files))
  mkdir -p -- ${(k)files} &&
    for dir (${(k)files}) mv -i -- ${(0)files[$dir]} $dir/

files above is an Associative array whose keys are the target directories, constructed from the first two IFS-delimited fields of the first line of each file matching file*(N.L+3) (whose name starts with file, that are regular (.) and have a Length greater than 3 (x y\n of size 4 is the smallest file that has a line with two fields)).

As a safeguard, we forbid ., .. or empty directory components or those containing /.

The value of the associative array elements is the list of files for a given target dir, NUL-delimited.

Then, we create all of those dirs at once and only if that succeeds start moving files in them.

Answer 3

I'm tempted to just do this (in Bash or any POSIX-y shell):

for f in ./*; do
    read -r a b < "$f"
    mkdir -p -- "$a/$b"
    mv -- "$f" "$a/$b"
done

That is, loop over the files, read a line, splitting it to two fields, make a directory based on those fields, and move that file to that directory. The -p option to mkdir creates parent directories as necessary and ignores existing dirs.

Yes, this does extra work in that it calls both mkdir and mv once for each file, and yes, I'm assuming your files don't contain strings like /dev/foo. But it should work and didn't take too long to write.

Running that on your sample files gives:

$ ls -R
.:
apple/  grapes/  orange/  pear/

./apple:
1/  3/

./apple/1:
file1

./apple/3:
file5  file6

./grapes:
3/
[...]