0

I have a pipe delimited file as below.

data.txt

ESP|041336|46566|NY|CA
ESP|041337|46566|NY|CA
ESP|041338|46566|NY|CA
ESP|041339|46566|NY|CA
ESP|041340|46566|NY|CA
ESP|041341|46566|NY|CA

I have another file with values in the second column that needs to be removed.

Input.txt

041337
041338
041339

I am trying to remove rows that have values from input.txt in its second column.

Expected Output

ESP|041336|46566|NY|CA
ESP|041340|46566|NY|CA
ESP|041341|46566|NY|CA

I am trying to use grep here as below to achieve this.

grep -vfw input.txt data.txt > output.txt

This doesn't remove the columns but throws an error No such file or directory and returns an empty file.

Improve this question
1

3 Answers 3

Reset to default
2

If you want to limit the comparison to the second delimited field only, then you can use awk instead of grep:

$ awk -F'|' 'NR==FNR {a[$1]++; next} !($2 in a)' input.txt data.txt > output.txt

$ cat output.txt
ESP|041336|46566|NY|CA
ESP|041340|46566|NY|CA
ESP|041341|46566|NY|CA
Improve this answer
0
2

The option -f expects a file name. The way you write it, the file name is the text following the f, in -vfw, in other words it is w.

You have to put an option that takes an argument at the end (except for tar, that uses a strange option parsing).

Any of these should work.

grep -v -w -f input.txt data.txt > output.txt
grep -vwf input.txt data.txt > output.txt
grep -vwfinput.txt data.txt > output.txt

Note: the exact error message, that you should always quote in full when you ask a question, mentions the name w.

grep: w: No such file or directory

Improve this answer
2
  • Thanks. It works. I think it looks like the whole line for the value in input.txt. Is there any way I can I make it look at only column 2 in data.txt? Commented Sep 3, 2018 at 22:31
  • 1
    Grep is designed to work on the whole line. With some transformation you can create patterns that only match the second field, Awk yould be a better tool, the example in the other answer should do what you want. Commented Sep 3, 2018 at 22:37
1 
sed -nE '
   /\|/!{H;1h;d;}
   G
   /^[^|]+\|([^|]+)\|.*\n\1(\n|$)/!P
'  input.txt data.txt

Result:

ESP|041336|46566|NY|CA
ESP|041340|46566|NY|CA
ESP|041341|46566|NY|CA

Comments:

  • Store the input.txt lines in the hold space by virtue of not having a pipe in it. /\|/! These lines are separated by newlines in the hold space.

  • For the data lines, we append the hold area to the pattern space, then try to look for the second pipe delimited field.  If there isn't one to be found, then print the portion before the first newline in the pattern space P.

    The regex, /^[^|]+\|([^|]+)\|.*\n\1(\n|$)/ matches the pattern space if the second |-delimited field in the data line (from data.txt) matches one of the input strings, which are at the end of the pattern space, delimited by newlines.  It has an OR condition at the end, to take care of the scenario when the matching pattern was the last line of the input.txt file.

Assumes a GNU sed version with extended regexps enabled to reduce the line noise.

Improve this answer
1
  • Please explain.  Please do not respond in comments; edit your answer to make it clearer and more complete. Commented Sep 4, 2018 at 4:32

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.