See if any files of a given extension exist within folder structure, excluding a few directories

Question

I'm trying to put a script together that i can use in my CI build that will work as a linter for my Typescript Mono-repo project.

What I'm trying to accomplish is see if there are any .js files within the project/package repos, but avoid node_modules folders where ever they are located and also the common folder and all its subdirectories and files.

The file structure is like this

project/
├── apps/
│   ├── app1
├── common/
│   ├── folder1
│   ├── folder1
│   ├── etc
├── lib/
│   ├── lib1
├── services/
│   ├── service1

With the sub-directories of services,lib,apps all being npm projects.

Currently some of the different attempts I've tried include:

find . \( ! -name node_modules \) -o \( ! -path common -prune \) -name "*.js" -print

But it doesn't seem to return anything even when I create dummy .js files in /project or any of the other whitelisted folders.

This attempt seems to be the closest as it prints out the test files, but it also finds files within common, but it should be blacklisted.

find . -type d \( -path common -o -name node_modules \) -prune -false -o -name '*.js'

The output of that run:

./test.js
./common/temp/pnpmfile.js
./common/temp/pnpm-store/2/registry.npmjs.org/ipaddr.js
./common/config/rush/pnpmfile.js
./common/scripts/install-run.js
./common/scripts/install-run-rushx.js
./common/scripts/install-run-rush.js
./services/test.js

The overall goal I want from this script is to, exit 0 if no .js files are found within the whitelisted directories or return exit 1 if a .js file is found outside of the blacklisted directories.

I'm pretty new and inexperienced with shell commands, so any help would be appreciated Thank you in advance!

Answer 1

Your command is almost correct. Fixing it:

find . \( -name node_modules -o -path ./common \) -prune -o -name '*.js' -print

This prunes anything called node_modules or that is located at ./common. Note that the pathname used with -path is a path with the top-level search path as start, and therefore must start with . in this case.

The -false that you use is not needed. It would be needed if either node_modules or ./common had also matched your -name test after -o. This is also when you may have wanted to use -type d, which you now don't have to do.

Testing:

$ tree
.
|-- common
|   `-- file.js
|-- dir
|   |-- common
|   |   `-- file.js
|   |-- file.js
|   `-- node_modules
|       `-- file.js
`-- node_modules
    `-- file.js

5 directories, 5 files

$ find . \( -name node_modules -o -path ./common \) -prune -o -name '*.js' -print
./dir/file.js
./dir/common/file.js

The file ./dir/common/file.js is found since it lives in a non-pruned directory (common is only pruned if it is directly below the current directory).

---

Elsewhere, it seems that all you want to do is to see if there are any files matching *.js in the non-pruned directories. You can do this efficiently with

if find . \( -name node_modules -o -path ./common \) -prune -o -name '*.js' -exec echo . \; | grep -q .
then
    echo there are JS files
else
    echo there are no JS files
fi

Here, I output a single dot (could be any character) whenever I find a file, and I use grep -q . to detect any output from find. When something is found, grep would terminate with a zero exit-status, signalling "success", and the if statement would take the first branch. The find command would terminate as soon as it tried to write another dot to the pipe that grep no longer is listening to.

Answer 2

If you use -path use a fully declared path like ./common or use -name to match anywhere in your found tree.

This command worked for me as a test.

find . -type d \( -name common -o -name node_modules \) -prune -false -o -name '*.js'

Answer 3

The command I ended up going with after reviewing and testing some of the above answers is this:

find . \( -name node_modules -o -path ./common \) -prune -o -name '*.js' -print

And the reason I had to keep the -print arg is that I found the output I got when running it without -print was showing some of the folders as output like this:

./common
./lib/trader-sdk/node_modules
./lib/is-ticker-symbol/node_modules
./apps/cron-app/node_modules
./services/templating-service/node_modules
./services/api-service/node_modules
./services/trading-services/node_modules
./services/sentiment-service/node_modules

But when I tacked on the -print arg the output was nothing, which is what was expected in this situation.

Bonus: This is the final use case, a basic script that will check my TypeScript mono-repo to ensure that no-one is trying to merge any build .js files into master.

#!/bin/bash

jsFiles=$(find . \( -name node_modules -o -path ./common \) -prune -o -name '*.js' -print)

if [ -z "$jsFiles" ]
then
      echo "No JS files found, free to proceed"
      exit 0
else
      echo "JS files found, unable to proceed"
      exit 1
fi