3

I have a file that is a mapping between filename and the corresponding transcripts. The file name and transcripts is separated by a TAB character.

The transcription of the file may contain one or more words that are separated by a single blank space. Following is a layout of the file,

[filename] [tab space] [trancription]

In some lines, the transcript column is empty. Such lines will be of the form

[filename]

i.e. there is no transcript available for that filename.

Now, my job is to make sure that I select only those lines that have a filename and transcription (that is the files whose transcript column is not empty).

I tried the following commands

(1) awk 'NF>2' filename

(2) awk 'NF==2' filename

(3) awk 'NF>1' filename

but did not get the results

In addition, when I used the command

(4) awk ' NF==2 {print $0} '   myfile  > newfile

I was also getting those lines with had only one column namely the "filename" field.

When I write NF<1 there is no output (as expected) When I write NF<2 again there is no output (strange, it should have displayed lines with only one column) When I write NF ==3 I am getting those lines which have exactly two columns (again confusing)

What's the catch? Its really confusing.

Now I am sending you the input sample

M07UP36A0821I40.wav 
M07UP36A0821I41.wav 
M07UP36A0821I410.wav    gaajara <bn>
M07UP36A0821I411.wav    tiina sau <pau> taintaaliisa
M07UP36A0821I412.wav    geehuun anya <bn>
M07UP36A0821I413.wav    geehuun daraa <babble>

Now I use the command 

grep '^[^[:blank:]]\+[[:blank:]]\+[^[:blank:]]\+$' cll

This command is giving no output (neither on the terminal nor in the redirected file).

Now there is an INTERESTING thing to note:

When the input file contains 

M07UP36A0822I413.wav    <bn> geehuun daraa <horn> <babble>
M07UP36A0822I414.wav    
M07UP36A0822I415.wav    gudxqa piilaa <horn> <babble>
M07UP36A0822I416.wav    <vn> gudxqa
M07UP36A0822I417.wav    gudxqa
M07UP36A0822I418.wav    gudxqa anya <babble>
M07UP36A0822I419.wav    harii matxara <bn> <babble>

Again, on using the same command

grep '^[^[:blank:]]\+[[:blank:]]\+[^[:blank:]]\+$' foo

terminal has STARTED showing the output. The output in this was obtained as

M07UP36A0822I417.wav    gudxqa

My desired output for the file foo would be those lines that are complete (both first column and second column must be there) Here is the required output

M07UP36A0822I413.wav    <bn> geehuun daraa <horn> <babble>
M07UP36A0822I415.wav    gudxqa piilaa <horn> <babble>
M07UP36A0822I416.wav    <vn> gudxqa
M07UP36A0822I417.wav    gudxqa
M07UP36A0822I418.wav    gudxqa anya <babble>
M07UP36A0822I419.wav    harii matxara <bn> <babble>

I used the following command on the file cll (the first sample in my question)

awk -F'\t' '(NF !=2) { print "line: " NR " does not have 2 columns: " $0 ;}' cll

The result was displayed on the terminal. The result is 

line: 1 does not have 2 columns: M07UP36A0821I40.wav 
line: 2 does not have 2 columns: M07UP36A0821I41.wav 
line: 3 does not have 2 columns: M07UP36A0821I410.wav    gaajara <bn>
line: 4 does not have 2 columns: M07UP36A0821I411.wav    tiina sau <pau> taintaaliisa
line: 5 does not have 2 columns: M07UP36A0821I412.wav    geehuun anya <bn>
line: 6 does not have 2 columns: M07UP36A0821I413.wav    geehuun daraa <babble>
Improve this question
6
  • 1
    "I tried ___ but did not get the results." What is that supposed to mean? What results did you get, exactly? The best thing to do on these types of questions is to give a sample of input and expected output. Commented Feb 8, 2016 at 7:46
  • 1
    Please edit with a sample of your real input file. awk 'NF==2' ought to be a correct solution to your problem as described, so something else is going on here. Commented Feb 8, 2016 at 8:00
  • "The best thing to do on these types of questions is to give a sample of input and expected output" . Is it possible to upload input file and the output file on this forum so that you could have a better look at it? Commented Feb 8, 2016 at 8:06
  • 1
    Just copy and paste a representative extract from the input file into the question, and put it in a code block (indented four spaces, like what I did in my edit). Commented Feb 8, 2016 at 8:10
  • I have given one small portion of the input file (see the modified question). This produced nothing on the output Commented Feb 8, 2016 at 9:15

3 Answers 3

Reset to default
4

Using grep:

grep -E '^[^\s]+\s+[^\s]+$' file.txt

  • [^\s]+ will contain the first column, then any number of white spaces \s+, followed by the second (last) column

  • ^ indicates the start and $ indicates the end of the line

POSIX-ly:

grep '^[^[:blank:]]\+[[:blank:]]\+[^[:blank:]]\+$' file.txt
Improve this answer
7
  • When I use this command on the terminal, there is no output(no text at all on the terminal). However doing wc -l on the output gives the output 539. Then I tried to redirect the output of this command (grep command that you mentioned) to some text file, I got the results. Why was there nothing to display on the terminal? Also, when I redirected the output to a file, I got getting some lines with empty second column. Commented Feb 8, 2016 at 7:59
  • @Snowbell check the second one.. Commented Feb 8, 2016 at 8:06
  • grep '^[^[:blank:]]\+[[:blank:]]\+[^[:blank:]]\+$' file.txt. This command is behaving in the same manner as the first one(no output at terminal; output is displayed only when command output is redirected to some file). Moreover, this command produces a files with (116) lines only. Also, there were only few lines with second column that was not empty. Commented Feb 8, 2016 at 8:10
  • @Snowbell Please edit your question to add some example text so that we can verify this.. Commented Feb 8, 2016 at 8:14
  • I have added two samples of my input file and the corresponding output that I received... Commented Feb 8, 2016 at 9:24
0

Other answers, using \s+ are over-matching! \s matches spaces, too, which occur in the data. What you are looking for is "Field_not_containing_TAB" followed by "Tab", followed by "Field_not_containing_TAB". or, in regexp-speak,

egrep '^[^Tab]+Tab[^Tab]+$' file.txt

Note: there should be no Space characters between the ''s in my egrep rexexp. You see them there due to my use of the kbd formatting tag, which I had to use to make Tab visible.

The regexp matches (or with the -v option to egrep, does not match) the supplied data.

Improve this answer
1
  • In this case, whatever is the input file, the same is the output file. NO changes at all.....I used this command on the 'cll' file (as described in my question) Commented Feb 8, 2016 at 10:52
0

For you, 2 columns is: something[TAB]something, something can have a space in it. So just tell awk your Field Separator is a tab:

awk -F'\t' '(NF==2)'

to find out the others : 

awk -F'\t' '(NF !=2) { print "line: " NR " does not have 2 columns: " $0 ;}'

Edit : you seem to "mislead" about the file's structure : you said it will contain either 1 word, or 1 word, a TAB, and 1 or several words (separated with spaces). If so, the above should have worked. However, given the result of the above command on your file, I guess it is instead: 1 word, [spaces or tabs], and maybe words[and spaces or tabs in between]

Thus the solution becomes:

awk -F'[\t ]+'  '(NF>=2)'    #words are separated by space&tabs, and we want at least 2 words [a filename, and its description]

a variant: outputs any line that have the 2nd word non empty:

awk -F'[\t ]+'  '(length($2)>0)'    #words are separated by space&tabs, and we want athe 2nd word to be non empty
#or, one that also verifies the 1st word is non empty:
awk -F'[\t ]+'  '(length($1)>0 && length($2)>0)'    #1st and 2nd words are non-empty
Improve this answer
7
  • I used the command. I have put the results that I got in the question...kindly see it Commented Feb 8, 2016 at 10:09
  • if it didn't work, it may be because you don't have just a tab ? or no tab at all? Commented Feb 8, 2016 at 10:52
  • When you will copy the content of cll file in a gedit file, the tabs that are there no longer appear in the gedit file. You have to manually put the tabs before executing any command. I tried this also.....The command awk -F'\t' '(NF !=2) { print "line: " NR " does not have 2 columns: " $0 ;}' shows no output at all Commented Feb 8, 2016 at 11:10
  • @Snowbell : It didn't show any output because it contained always 2 fields separated by 1 tab. However, try the latest version : it should work if you have TABs or if you have SPACES, in both cases. Commented Feb 8, 2016 at 12:01
  • awk -F'[\t ]+' '(length($2)>0)' This command is working. Thanks. could you please explain "[\t ]+" in the command Commented Feb 9, 2016 at 6:46

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.