Counting file extensions in a string array

Question

I have a $FILES array which consists of strings of 4 filenames:

FILES=$(ls /my/directory | shuf -n 4)

The contents of the array look something like: file1.jpg file2.jpg file3.gif file4.mp4

I want to count how many of each file types there are. Currently I am doing this:

nrofimg=$( grep -o .jpg <<< ${FILES[*]} | wc -l )
nrofgif=$( grep -o .gif <<< ${FILES[*]} | wc -l )

but this is not optimal, because if a file is called file.jpgfile.jpg, it will count it twice. What is a better error-proof precise way of counting how many of each file extensions there are in the array?

EDIT: This seems to work:

nrofjpg=$( grep -c '\.jpg$' <<< ${FILES[*]} )

the grep -c and '\.jpg$' seems to do what I want...

EDIT 2: Later in the code I am checking file extensions in a loop with a if statement, and files with spaces are seen as 2 files. This is not good, the double quotes around $f do not help.

for f in $FILES; do
    if [[ "$f" == *.jpg || "$f" == *.png || "$f" == *.jpeg || "$f" ==   *.bmp ]]; then
        ..code..
    fi
done

Answer 1

You can't use line-based text utilities to process file names as newline is as valid a character as any in a file (file names don't even have to be made of text).

In zsh, you can get non-hidden files names at random into an array with:

rand() ((REPLY=RANDOM))
files=(/my/directory/*(No+rand[1,4]:t))

We're using shell globbing instead of ls to get the list of files. Here using the o+function glob qualifier to order the list of files based on the value returned by the function in $REPLY, while N enables nullglob for the one glob and :t gets the tail (basename) of each.

You can get the number of $files elements that Match the *.jpg glob with:

() {nrofjpg=$#} ${(M)files:#*.jpg}

With the bash shell, assuming version 4.4 or newer and with GNU utilities, you could do something approaching with:

print0() {
  (( $# == 0 )) || printf '%s\0' "$@"
}

readarray -td '' files < <(
  shopt -s nullglob
  cd /my/directory &&
    print0 * | shuf -zn 4
)

nrofjpg=$(print0 "${files[@]}" | grep -zc '\.jpg$')

With GNU ls 9.1 (released just over 3 hours after this question was asked) or newer, you can replace the:

  shopt -s nullglob
  cd /my/directory &&
    print0 *

With:

ls --zero /my/directory/

In any case, the point is to use NUL-delimited records instead of lines as 0 is the only byte value that can't occur in a file path.

To get a break down of number of files per extension, assuming that by extension you mean the non-. characters (or non-characters) between the right most occurrence of a . in a filename and the end of the file name, in zsh, you could do:

typeset -A count
for file ($files) (( ++count[\$file:e] ))

And you'd then get the number of jpg files in $count[jpg] for instance, or do things like:

(( nrofimg = count[jpg] + count[jpeg] + count[bmp] + count[png] ))

Note that $file:e expands to the empty string for both file and file.

Or in bash:

typeset -A count
for file in "${files[@]}"; do (( ++count[.\${file##*.}] )); done

And the count of jpg files being in "${count[.jpg]}". We prepend a . because bash associative arrays choke on the empty key (like in a file called file.). Note that a file without . such as file would be counted in "${count[.file]}".

---

Some other notes about your approach:

• FILES=$(ls /my/directory | shuf -n 4) is a scalar variable assignment. For an array assignment, the syntax is FILES=($(ls /my/directory | shuf -n 4))
• but in any case, with that fixed, the array elements will be the result of split+glob applied to ls' output which is not what you want here. readarray -t FILES < <(ls /my/directory/ | shuf -n 4) (another way to assign arrays as a whole) would be a bit less bad, but still be wrong as it doesn't handle filenames with newline characters properly.
• . is special in the regular expression syntax (grep). It matches any single character. So grep -o .jpg on ajpgbjpg.jpg would output ajpg, bjpg and .jpg.
• With grep -c ... <<< ${FILES[*]}, the elements of the $FILES array are joined with the first character of the $IFS special character, space by default, a newline is appended and that text is fed as standard input to grep, so grep will only ever get one line of input (unless the file names do contain newline characters themselves¹)
• All uppercase variable names are better reserved for environment variables. Arrays can't be exported to the environment in bash anyway.
• in bash, when the FILES variable is an array (or associative array), $FILES is the same as ${FILES[0]}. In that shell, which has copied the poor array design of the Korn shell, to loop over the elements of the array (the values of the elements for associative arrays), you need for file in "${FILES[@]}"; do.... for file in $FILES would be zsh syntax (though beware empty values are elided; not an issue here) or yash syntax (though beware values are subject to split+glob).
• instead of [[ "$f" == *.jpg || "$f" == *.png || "$f" == *.jpeg || "$f" == *.bmp ]], you can use case $f in (*.jpg | *.png | *.jpeg | *.bmp) (standard sh syntax), or in zsh: [[ $f = *.(jpg|png|jpeg|bmp) ]] or in ksh/bash: [[ $f = *.@(jpg|png|jpeg|bmp) ]] (you'll also need shopt -s extglob in older versions of bash).

---

^{¹ and your version of bash is not too old in which case the expansion would be split on all characters of $IFS (which also includes TAB and newline by default) and joined back with the first character of $IFS}