How to use inotifywait to watch a directory for creation of files of a specific extension

Question

I have seen this answer:

You should consider using inotifywait, as an example:

inotifywait -m /path -e create -e moved_to |
    while read path action file; do
        echo "The file '$file' appeared in directory '$path' via '$action'"
        # do something with the file
    done

The above script watches a directory for creation of files of any type. My question is how to modify the inotifywait command to report only when a file of a certain type/extension is created (or moved into the directory). For example, it should report when any .xml file is created.

What I tried:

I have run the inotifywait --help command, and have read the command line options. It has --exclude <pattern> and --excludei <pattern> options to EXCLUDE files of certain types (by using regular expressions), but I need a way to INCLUDE just the files of a certain type/extension.

Answer 1

how do I modify the inotifywait command to report only when a file of certain type/extension is created

Please note that this is untested code since I don't have access to inotify right now. But something akin to this ought to work with bash:

inotifywait -m /path -e create -e moved_to |
    while read -r directory action file; do
        if [[ "$file" =~ .*xml$ ]]; then # Does the file end with .xml?
            echo "xml file" # If so, do your thing here!
        fi
    done

Alternatively, without bash,

inotifywait -m /path -e create -e moved_to |
    while read -r directory action file; do
        case "$file" in
            (*.xml)
                echo "xml file" # Do your thing here!
                ;;
        esac
        fi
    done

With newer versions of inotifywait you can directly create a pattern match for files:

inotifywait -m /path -e create -e moved_to --include '.*\.xml$' |
    while read -r directory action file; do
        echo "xml file" # Do your thing here!
    done

Answer 2

--include and --includei are actual options, in the master version of the binaries at least:

https://github.com/inotify-tools/inotify-tools/tree/master

Answer 3

Starting with inotifywait version 3.20.1 you can use an --include option.

inotifywait example with --include

Here's an example that uses --include to run a script when a wav file is added to a directory.

#!/bin/sh

# Usage
#
# Call the script with the directory you want to watch as an argument. e.g.:
# watchdir.sh /app/foo/
#
#
# Description:
# Uses inotifywait to look for new files in a directory and process them:
# inotifywait outputs data which is passed (piped) to a do for subcommands.
#
#
# Requirements:
# Requires inotifywait which is part of inotify-tools.
# e.g. yum install -y inotify-tools or apt-get install -y inotify-tools.
#
#
# See also:
# https://linux.die.net/man/1/inotifywait
# https://github.com/inotify-tools/inotify-tools/wiki#info
# (Might be interested in pyinotify too.)


echo "Watch $1 for file changes..."


# Be careful not to confuse the output of `inotifywait` with that of `echo`.
# e.g. missing a `\` would break the pipe and write inotifywait's output, not
# the echo's.
inotifywait \
  $1 \
  --monitor \
  -e create \
  --timefmt '%Y-%m-%dT%H:%M:%S' \
  --format '%T %w %f %e' \
  --include "\.wav" \
| while read datetime dir filename event; do
  echo "Event: $datetime $dir$file $event"
  echo "Now, we could pass $datetime $dir $filename and $event to some other command."

  echo "Here's the extensionless filename:" ${filename%.*}
  echo "And the extension if needed:" ${filename##*.}

  python3 /app/phono.py --custom $dir$filename $dir${filename%.*}.txt /tmp/${filename%.*}
done


# --format:
#   %T  Replaced with the current Time in the format specified by the --timefmt option, which should be a format string suitable for passing to strftime(3).
#   %w  This will be replaced with the name of the Watched file on which an event occurred.
#   %f  When an event occurs within a directory, this will be replaced with the name of the File which caused the event to occur. Otherwise, this will be replaced with an empty string.
#   %e  Replaced with the Event(s) which occurred, comma-separated.
#   %Xe Replaced with the Event(s) which occurred, separated by whichever character is in the place of 'X'.
#
# There's no --include option, but there's --exclude, which can be used to the same effect as an include.


# test the script by creating a file in the watched directory, e.g.
# touch /app/foo/file1.wav # will trigger the script
# touch /app/foo/file1.txt # will not trigger the script

Installing (or compiling) inotify-tools

inotifywait is part of inotify-tools. You can install it via the usual way, e.g.:

apt-get install -y inotify-tools

or

yum install -y inotify-tools

But if only older versions are available from the repos, you may want to compile from source. If so:

cd /tmp/inotify-tools/
wget https://github.com/inotify-tools/inotify-tools/releases/download/3.20.2.2/inotify-tools-3.20.2.2.tar.gz
tar xzvf inotify-tools-3.20.2.2.tar.gz
cd inotify-tools-3.20.2.2
./configure --prefix=/usr --libdir=/lib64
make
make install

Answer 4

Use a double negative:

inotifywait -m --exclude "[^j][^s]$" /path -e create -e moved_to |
    while read path action file; do
        echo "The file '$file' appeared in directory '$path' via '$action'"
    done

This will only include javascript files

Answer 5

Whilst the double-negative approach of the previous answer is a nice idea since (as TMG noted) it does indeed shift the job of filtering to inotifywait, it is not correct.

For example, if a file ends in as then it will not match [^j][^s]$ because the final letter s does not match [^s], therefore it will not be excluded.

In Boolean terms, if S is the statement:

"the final letter is s"

and J is the statement:

"the penultimate letter is j"

then the value of the --exclude parameter should semantically equate to not(J and S), which by De Morgan's laws is not(J) or not(S).

Another potential problem is that in zsh, $path is a built-in variable representing the array equivalent of $PATH, so the while read path ... line will completely mess up $PATH and cause everything to become unexecutable from the shell.

Therefore the correct approach is:

inotifywait -m --exclude "[^j].$|[^s]$" /path -e create -e moved_to |
    while read dir action file; do
        echo "The file '$file' appeared in directory '$dir' via '$action'"
    done

Note the . which is needed after [^j] to ensure that the match is applied in the penultimate position, and also that the | character (representing the boolean OR mentioned above) should not be escaped here because --exclude takes POSIX extended regular expressions.

However, please see and upvote @ericcurtin's answer which for newer versions of inotifywait is a far cleaner approach.

Answer 6

The best solutions are the ones that suggest using inotifywait's built in --include and --includei (from version 3.20.1).

Notice there is a bug with --include and --includei that result in relevant files being ignored when new directories are created while inotifywait is watching the parent directory. See the open Github issue. At the time of writing, version 3.22.6.0 has this bug.

A workaround is to validate the filename in the loop.

inotifywait \
  <DIRECTORY_TO_WATCH> \
  --monitor \
  --recursive \
  -e create \
  --timefmt '%Y-%m-%dT%H:%M:%S' \
  --format '%T %w %f %e' \
| while read datetime dir filename event; do
  # if the file does not include .xml then continue the loop
  ! [[ "$filename" =~ \.xml ]] && continue

  # otherwise the file includes .xml and we do something with it
  echo "Event: $datetime $dir$file $event"
done

This is not optimal, but the best solution if you will have directories being dynamically created while inotifywait is watching the parent directory.

Answer 7

bash interactive script monitor using inotifywait:

As all answer here address inotifywait -m | script, I would like to suggest another way to waiting for input... More flexible!!

Note: regarding issue with --include, I use case $file in *.xml which is one of the quickest doContainString test to be done.

Basically:

while read path action file; do
    case $file in
        *.xml )
                # Do something with "$file"
                printf 'XML File: %s/%s\n' "$path" "$file"
                ;;
    esac
done < <(inotifywait -me close_write,moved_to /path)

• Having main script reading input from redirected sub -process is easier to understand mostly about global variables en further script.
• Using close-write event ensure you file won't be modified at time your script will work on.

Interactive script

• Use another File descriptor for inotifywait will let you use STDIN for other interactions:

  exec {inotimon}< <(exec inotifywait -m /tmp/so -e close_write 2>&1)
  

  This will run inotifywait in background and redirect both STDOUT and STDERR to a new file descriptor, then store file descriptor's value to $inotimon variable.

• Use trap, kill and USR2 signal for avoiding pool on two inputs, so we run a simple read -rsn 1 char in background, then send character (with a newline) to a new descriptor ($userOut) and use kill -USR2 to interrupt main script.

#!/bin/bash

declare -i countXml=0
treatXml() {
    # Do something with "$file"
    printf 'XML File: %s/%s\n' "$1" "$2"
    countXml+=1
}
showStat() {
    printf 'There is %d files treated.\n' $countXml
}
userInput() {
    while IFS= read -d '' -rsn 1 char; do
        printf %q\\n "$char"
        kill -USR2 $1
    done
}
trapUI() {
    read -ru $userOut userKeyVal
    case ${userKeyVal,,} in
        s|*\\t* ) showStat ;;   # Keys ``s'' or ``Tab'' will show stats
        q|*\\e* ) exit ;;       # Keys ``q'' or ``Esc'' will quit.
    esac
}

exec {userOut}< <(userInput $$)
uipid=$!
exec {inotimon}< <(
    exec inotifywait -me close_write,moved_to /root/path 2>&1)
inotipid=$!
closeAll() {
    exec {inotimon}<&-
    exec {userOut}<&-
    kill $uipid $inotipid
}
trap trapUI USR2
trap closeAll 0 1 2 3 6 15
while read -ru $inotimon path action file; do
    case $action$file in
        CLOSE*.xml|MOVED*.xml ) treatXml "(${action,,}) $path" "$file";;
        e* ) printf 'FD %d: %s\n' $inotimon "${path^^} ${action^^}" ;;
    esac
done

Answer 8

In my case I was trying to filter the output of inotifywait using something like grep, in order to wait for the creation of a single file with a particular extension. The trouble with this approach is that grep will just hang.

For some reason a SIGPIPE signal is either not sent, or not processed by inotifywait. I found this question, which ran into the same issue I had, and process substitution can be used as a workaround:

grep -E .*iso -m1 < <(inotifywait -qm -e create .)

Process substitution does not work in certain shells (eg. sh), so this does not work in every case.

---

From the other answers given, I could not find the --include option. I will likely be experimenting with --exclude.