How to grep through multiple files using lines from two input files as start and end of pattern?

Question

I would like to write a simple script that would use grep to search through a list of files. The current code looks something similar to this:

a= file1.txt 

b= file2.txt

for

do

grep '$a.*$b' /root/listoffiles/*php >> found.txt

done

file1 and file2 are both word lists. So this would search through the destination having multiple criteria. For example $a could be "hello" and $b "world" or the second line could be "red" and "tshirt". If it finds anything with two matching criteria then it would just save it into the file. The loop that I am using is not working and not efficient enough. I would like to get only the results for meeting criteria. Any suggestions?

Answer 1

You would first have to escape the special characters in those two files (if any) and then join them via e.g. paste to end up with a list of patterns that you would then use via grep's -f ... and unless you have too many files you don't need a loop as grep accepts multiple arguments:

paste -d'.*' <(sed 's/[.[\*^$]/\\&/g' file1.txt) /dev/null \
<(sed 's/[.[\*^$]/\\&/g' file2.txt) | grep -f - /root/listoffiles/*php >> found.txt

Answer 2

Build a pattern from the word lists. From file1.txt containing e.g. one, un, eins and file2.txt containing two, deux, zwei, you need to get the regular expression (in ERE syntax)

(one|un|eins).*(two|deux|zwei)

If the word lists don't contain any characters that are special in a regex then all you need to do is replace newlines by | (the “or” operator) except the final one.

words1=$(tr '\n' '|' <file1.txt); words1=${words1%\|}
words2=$(tr '\n' '|' <file2.txt); words2=${words2%\|}
grep -E "($words1).*($words2)"

If the word lists might contain special characters then you need to quote them.

words1=$(<file1.txt tr '\n' '|' | sed 's/[][\\().*?+|]/\\&/g'); words1=${words1%\|}
words2=$(<file2.txt tr '\n' '|' | sed 's/[][\\().*?+|]/\\&/g'); words2=${words2%\|}
grep -E "($words1).*($words2)"

Note that if you don't care about the order of the words and you can accept overlapping matches (e.g. accept twone as containing both one and two) then all you need is two grep invocations:

grep -f file1.txt | grep -f file2.txt