5

Is it possible to use the multi-line output of a grep run of one file as patterns for use with a subsequent second grep run on another file?

Example:

  • content of file1.txt 
    2 blue
1 red 
1 green
2 black
2 orange
  • content of file2.txt 
    2 blue triangle
2 blue circle
3 blue triangle
2 red triangle
3 green circle
4 red square
2 orange circle
2 brown circle
  • Result of first grep: 
    $ grep 2 file1.txt 

2 blue 
2 black
2 orange

Now, I am looking for a command that implements something like

grep <PREVIOUS OUTPUT> file2.txt

that would find all lines in file2.txt that start with one of the lines produced by the grep run on file1.txt, so the desired result is:

2 blue triangle
2 blue circle
2 orange circle
Improve this question
5
  • It strongly depends, so we would need more information to propose a definite solution. It seems you are working with table-based data. So, are the separators guaranteed to be always "one space"? Can the pattern 2 blue appear anywhere else in file2.txt "by chance" (as in 4 red square and 2 blue triangles), i.e. do you need full RegEx search for matching the lines in file2.txt against the output of the first grep? Are other standard text-processing tools also acceptable, or does it need to be grep? Commented May 10, 2023 at 12:38
  • 1
    If you have found a solution yourself, please don't edit your question to include it. Instead, add you own answer (as you already did). Commented May 10, 2023 at 12:48
  • 1
    It's not clear if you want a regexp or string comparison (your example only contains literal chars but we don't know if that's an oversight), nor if you want a partial match or full match, e.g. should grep 2 on the first grep match 21 red or 12 green from the first file? should 2 blue from file1 match 2 blueish-green from file 2. Commented May 10, 2023 at 20:10
  • grep -E "$(grep 2 file1.txt| awk '{ORS="|"; print}'| head -c-1)" file2.txt Commented May 11, 2023 at 6:05
  • Are you willing to use sed, awk, perl, etc.? Commented May 15, 2023 at 20:24

2 Answers 2

Reset to default
15

This is the answer to the literal question. For an answer that is likely better to solve the task, see the awk solution in the "Notes" below.

Assuming that the patterns found in file1.txt can only ever occur at the beginning of lines in file2.txt, you can use the -f flag to read multiple search patterns from a file rather than stating them as RegExes on the command-line. The file would then need to be connected to the output of your first grep run.

One way to do this is process substitution:

grep -F -w -f <(grep -w 2 file1.txt) file2.txt
  • The <( ... ) construct makes the output of the command enclosed in parentheses available as if it were a file.
  • The -F flag disables full RegEx search and is for safety and in case the output of the first grep run can contain characters with special meaning in the context of RegExes. It also speeds up the matching as literal string comparison is faster then RegEx matching.
  • The -w flag ensures that no partial matches find their way into the results. This is particularly advisable for the first grep run in case the first column of file1.txt can contain multi-digit numbers such as 12.

Update

As noted by @Stéphane Chazelas, the -f option accepts the often (but not always) implemented value of - to refer to the program's stdin, so you can also write this as

grep -w 2 file1.txt | grep -F -w -f - file2.txt

with the more easily recognized pipe-method to read from the output of another command.

Note

  1. This assumes the pattern number color can only occur as the first two columns of file2.txt. If it can also occur later on the line, as in

    4 red square and 2 blue triangle

    such a line would be misidentified as also matching.

  2. For working with tabulated data (as seems to be the case here), awk is usually a better-suited tool. Your task could be achieved by the following awk program:

    awk -v num="2" 'NR==FNR{if ($1==num){col[$2]}; next} ($1==num) && ($2 in col)' file1.txt file2.txt

    This will process both files. The search key is specified via an awk variable num.

    While processing the first file (where NR, the global line counter, is equal to FNR, the per-file line counter), it registers the color in the second column in the indices of an array col (hence no actual assignment to col[$2]) if the number in the first column matches. It then skips execution to the next line.

    When processing the second file, it checks if the first column matches the num variable, and if the second column is found among the indices of the array col. If so, the line is printed.

Improve this answer
0
0

Solved:

$ grep 2 file1.txt | xargs -I{} grep {} file2.txt

2 blue triangle
2 blue circle
2 orange circle
Improve this answer
1
  • 6
    Note that xargs will choke on quotes or backslash characters and remove leading blanks. grep will also choke on strings starting with - (you'd need -- or -e) and will treat them as regexp (you may want a -F). So grep 2 file1.txt | xargs -d '\n' -I {} grep -Fe {} file2.txt (assuming GNU xargs), though that'd still be quite inefficient as it runs on grep per matching line of file1.txt, each reading file2.txt in full. Commented May 10, 2023 at 16:52

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.