inotifywait ignoring new folders in watch directory

Question

I am trying to use inotifywait to watch a folder (/shares/Photos) and when it detects a jpg added to the folder I need to resize it into a sub directory ($small_dir). Under the photos director there will be many subfolders for the jpgs.

The tree looks like this

shares
  -Photos
      -Folder 1
      -Folder 2
       .
       .
       .

Basically whenever someone copies pictures into folder 1 I need to create a new subfolder, and then resize the images and put the smaller versions into that folder.

So the tree would become:

shares
  -Photos
      -Folder 1
          -Resized
      -Folder 2
       .
       .
       .

My code so far:

inotifywait -mr --timefmt '%m/%d/%y %H:%M' --format '%T %w %f' -e close_write /shares/Photos --includei "\.jpg|\.jpeg" |
while read -r date time dir file; do
    changed_abs=${dir}${file}
    small_dir=${dir}${target}/
    
    printf "\nFile $changed_abs was changed or created\n  dir=$dir \n  file=$file \n  small_dir=$small_dir \n"
    
    # Check if the $small_directory exists, if not create it.
    if [ -d "$small_dir" -a ! -h "$small_dir" ]
    then
        echo "$small_dir found, nothing to do."
    else
        echo "Creating $small_dir"
        mkdir $small_dir
        chmod 777 $small_dir
    fi
    
    # Check to see if the file is in $small_dir, if it is, do nothing.  
    if [ "$dir" = "$small_dir" ]; then
        printf "\nFile is in the $small_dir folder, nothing to do\n"
    else
        printf "\nResizing file into the $small_dir folder\n"
        # Code to resize the image goes here.
    fi
        
done

It mostly works, but what I am banging my head against the wall about is that if I create a new subfolder under Photos while the script is running, inotifywait simply ignores it and does nothing.

I tried replacing close_write with create but it made no difference, and I am really not sure where to go from here.

Any advice/help would be greatly appreciated.

Answer 1

OP is using:

inotifywait -mr --timefmt '%m/%d/%y %H:%M' --format '%T %w %f' -e close_write /shares/Photos --includei "\.jpg|\.jpeg" |

The documentation about --includei tells (bold emphasis mine):

--includei <pattern>

Process events only for the subset of files whose filenames match the specified POSIX regular expression, case insensitive.

That's not: "display events" but "process events". Indeed that means that only events about directories with a name including .jpg or .jpeg will be processed.

A directory creation event occuring but not matching the filter won't be processed, so inotifywait will not call inotify_add_watch(2) on this event and later anything happening in this . As a result, there will never be events watched in this subdirectory.

I couldn't find with --includei or other similar options a way to express "process events only for these regex, or also with any directory".

---

UPDATE: suggest a workaround

So the way to get it working appears to have to filter outside of the command. GNU grep will buffer its output if not a tty, so add --line-buffered.

This will be affected by user input (like spaces in filenames). To mitigate this, using a / separator (which is invalid in a filename) between the directory and filename is needed. As the directory part conveniently includes a trailing /, just removing the space in the format string is enough (along further variable processing and reuse like changed_abs). At the same time, I'm correcting the intent to filter filenames ending with the strings jpg or jpeg, not including these strings, and probably improving initial handling of directories with space inside (affectations of changed_abs and small_dir but there are more later to fix). OP should really protect all relevant variables with quotes in the script ({ } doesn't replace quotes).

Replace:

inotifywait -mr --timefmt '%m/%d/%y %H:%M' --format '%T %w %f' -e close_write /shares/Photos --includei "\.jpg|\.jpeg" |
while read -r date time dir file; do
    changed_abs=${dir}${file}
    small_dir=${dir}${target}/

with:

inotifywait -mr --timefmt '%m/%d/%y %H:%M' --format '%T %w%f' -e close_write /shares/Photos |
    grep --line-buffered -Ei '/[^/]*\.(jpg|jpeg)$' |
while read -r date time changed_abs; do
    [ -d "$changed_abs" ] && continue # a directory looking like a picture filename was written to: skip this event
    dir="${changed_abs%/*}/"
    file="${changed_abs##*/}"
    small_dir="${dir}${target}/"

Not completely tested, but that's the idea. I'm not even sure the directory test is needed (it appears there is never a close_write event on it), but it won't hurt.

---

Notes

• if this isn't clear, an action (inotify_add_watch(2)) has to be done by inotifywait for each directory creation event detected, inotifywait must as soon as possible add a new watch to this directory because it might lose following events inside it (race condition). It's even documented in the BUGS section:

  There are race conditions in the recursive directory watching code [...] probably not fixable.

• newer versions of inotifywait, when running as root (or privileged enough) on kernel >= 5.9 at least, are supposed to be able to use the fanotify(7) facility, but I couldn't manage to get my version of inotifywait to use this despite being supposed to be compiled with support and having a recent enough kernel. On a system where fanotify(7) would be used, combined with the --filesystem option, hypothetically this could remove the need to have to do an action for each newer directory and make OP's method based on filtering with --includei work.