1

I have a directory with 2000+ text files. I am trying to make a script that:

  1. Reads a list of IP addresses from ip.txt
  2. Cats each file in the directory
  3. Greps each file for the IP address

If keyword is found, echoes the keyword and the file name to a file.

The output should be like this:

$ cat
results.txt
192.168.2.3 was found in 23233.txt
192.168.4.0 was found in 2323.txt

At the moment I have this:

while read p; do
for filename in *.txt; do
if cat $filename | grep "$p" 
then echo "$p" is  "$filename" | tee result.txt
fi
done
done<ips.txt

However this also echoes all file names into the results. How can I fix this?

Improve this question

3 Answers 3

Reset to default
4

First, save a cat by not using one when you don't need it. Rather than:

cat haystack | grep needle

You can simply:

grep needle haystack

As for your script:

> results.txt  # start with a fresh file for every run
while read ip; do
    grep "$ip" *  | grep -Ev 'results\.txt|ips\.txt' >> results.txt
done < ips.txt

The grep-into-grep pipeline is to prevent adding entries from the input and output files into the output file.

If you have a zillion files to check and you're getting argument list too long, we can use a tool like xargs to break our command up into chunks short enough for the shell to permit:

> results.txt  # start with a fresh file for every run
while read ip; do
    find . -type f -maxdepth 1 -not -name ips.txt -not -name results.txt -print0 | xargs -0 grep "$ip" >> results.txt
done < ips.txt

Here we're filtering out the input and output files with logic fed into find, so we no longer need to grep into grep.

Improve this answer
2
  • Thanks, i am getting the error 'grep argument list too long' this is why i originally did a loop. ( i should of mentioned) Commented Apr 15, 2019 at 16:09
  • Solution that can handle a zillion files has been provided. Commented Apr 15, 2019 at 16:17
0

Assuming your file only has IPv4 addresses (no IPv6), you can just run something like the following:

find [dir1] -maxdepth 1 -type f -iname ip.txt -exec grep -H '[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*' {} \;

This should handle an unlimited quantity of files. Your output will look something like [dir1]/ip.txt:1.2.3.4. Of course, you'll need to replace "[dir1]" with an actual directory (or you can omit this option, in which case find will just use your current working directory). You can specify multiple directories if you wish. In fact, there shouldn't be any real limit as to how many directories you can put in the find command, as long as you put them before the "-maxdepth" argument.

Improve this answer
0

Another option, if you're not married to grep for this, is my favorite tool I've adopted to replace grep long ago - ack. It can pretty much drop into your provided script and provide the output you need. (It is searching recursively by default, worth noting if you don't want files in subfolders searched).

while read p; do
  for filename in $(ack -l $p); do
    echo "$p found in $filename" >> results.txt
  done
done<ips.txt
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.