1

I have a text file comprising two tab delimited columns. I wish to grep each line (i.e. a character string comprising two tab delimited values) in another file. The first file has an empty line at the end. If I delete the empty line, the command below works fine. But if I don't, it does not.

I can delete the line, but would like to know why it fails. Also, my attempts to get round the problem without deleting the line also fail. Please help? Thanks.

This command works fine if the empty line is deleted:

grep -wF -f file1 file2 > file3

I have tried to get around the problem without deleting the line, by trying things like

grep '.' file1 | grep -wF - file2 > file3

But can't get it to work.

Improve this question
5
  • 1
    Empty line is a zero-length pattern, so it matches every line I guess. Commented Apr 17, 2021 at 18:28
  • In second command you forgot -f, should work like this: grep . file1 | grep -wF -f - file2 > file3 Commented Apr 17, 2021 at 18:44
  • Thank you both for your help. I messed up by failing to say the problem is that the grep returns no results (not all results, as you both assumed). I played around a bit more trying to generate a reproducible example, and I can see that my commands and the ones you suggested work fine on simplified data. Commented Apr 17, 2021 at 21:05
  • So I think the problem is that I generated file1 (the list of terms to grep) using write.table function in R, and this adds the empty line at the end of the file. But in fact, I can only see this empty line when I open file1 with a Windows text editor like Notepad++. If I open file1 using Vi in Bash, or look at it using tail file1, no empty line is apparent. So I guess R has put something at the end of the file which messes things up...? Commented Apr 17, 2021 at 21:06
  • @Esteban, then you're doing something other than just removing the empty line. Also, note that text files are supposed to end with a newline character, but some editors allow moving the cursor "below" the last line (that makes it easier to add a new line by just going there and typing, instead of having to go to the end of previously-last line just to hit enter, needing a separate command for adding the new line). Since you mention Windows, you might want to check that you're not trying to have grep read patterns off a file with Windows line endings, i.e. with the extra carriage returns. Commented Apr 18, 2021 at 16:11

1 Answer 1

Reset to default
2

An empty pattern matches any input line, so if your list of patterns in file1 contains one, then all lines in file2 will match. You didn't say, but I expect that's the problem you get, at least I can't think of any other issues an empty line would cause.

Your grep pipeline looks fine, except that you need the -f option to the latter grep:

grep '.' file1 | grep -wF -f - file2 > file3
                          ^^

Alternatively, you could use a process substitution:

grep -wF -f <(grep '.' file1) file2 > file3

That's a Ksh/Bash/Zsh feature: the <( cmd ) is replaced with a filename that produces the output of cmd when read.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.