Skip to main content
added 545 characters in body
Source Link
DopeGhoti
  • 79.2k
  • 10
  • 107
  • 141

First, save a cat by not using one when you don't need it. Rather than:

cat haystack | grep needle

You can simply:

grep needle haystack

As for your script:

> results.txt  # start with a fresh file for every run
while read ip; do
    grep "$ip" *  | grep -Ev 'results\.txt|ips\.txt' >> results.txt
done < ips.txt

The grep-into-grep pipeline is to prevent adding entries from the input and output files into the output file.

If you have a zillion files to check and you're getting argument list too long, we can use a tool like xargs to break our command up into chunks short enough for the shell to permit:

> results.txt  # start with a fresh file for every run
while read ip; do
    find . -type f -maxdepth 1 -not -name ips.txt -not -name results.txt -print0 | xargs -0 grep "$ip" >> results.txt
done < ips.txt

Here we're filtering out the input and output files with logic fed into find, so we no longer need to grep into grep.

First, save a cat by not using one when you don't need it. Rather than:

cat haystack | grep needle

You can simply:

grep needle haystack

As for your script:

> results.txt  # start with a fresh file for every run
while read ip; do
    grep "$ip" *  | grep -Ev 'results\.txt|ips\.txt' >> results.txt
done < ips.txt

The grep-into-grep pipeline is to prevent adding entries from the input and output files into the output file.

First, save a cat by not using one when you don't need it. Rather than:

cat haystack | grep needle

You can simply:

grep needle haystack

As for your script:

> results.txt  # start with a fresh file for every run
while read ip; do
    grep "$ip" *  | grep -Ev 'results\.txt|ips\.txt' >> results.txt
done < ips.txt

The grep-into-grep pipeline is to prevent adding entries from the input and output files into the output file.

If you have a zillion files to check and you're getting argument list too long, we can use a tool like xargs to break our command up into chunks short enough for the shell to permit:

> results.txt  # start with a fresh file for every run
while read ip; do
    find . -type f -maxdepth 1 -not -name ips.txt -not -name results.txt -print0 | xargs -0 grep "$ip" >> results.txt
done < ips.txt

Here we're filtering out the input and output files with logic fed into find, so we no longer need to grep into grep.

Source Link
DopeGhoti
  • 79.2k
  • 10
  • 107
  • 141

First, save a cat by not using one when you don't need it. Rather than:

cat haystack | grep needle

You can simply:

grep needle haystack

As for your script:

> results.txt  # start with a fresh file for every run
while read ip; do
    grep "$ip" *  | grep -Ev 'results\.txt|ips\.txt' >> results.txt
done < ips.txt

The grep-into-grep pipeline is to prevent adding entries from the input and output files into the output file.