1

Here is an example text file:

A B C D E F G
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9

I would like to extract specific columns based on the values on the 3rd row, that is, 2 3 4 5 6 7 8. Let's say, I would like to extract all the columns with a value on the 3rd row larger than 5. That will be the final 3 columns. Hence, my goal is to select and generate the following:

E F G
5 6 7
6 7 8
7 8 9

Here is my code:

NR==3 {
    for (i=1; i<=NF; i++) {
        if ($i > 5)   x[j++] = i
    }
}
NR>= 1 {
  for (i=0 ;  i < j-1; i++ )
     printf("%s ",$x[i])
  printf("%s\n",$x[j-1])
}

However, this generates the following:

A B C D E F G
1 2 3 4 5 6 7
6 7 8
7 8 9

What did I miss?

Improve this question
8
  • 1
    your problem is that lines 1 and 2 are processed before line 3, and go through the NR>=1 code block, with no initialization for j and x Commented Apr 4, 2013 at 15:49
  • 1
    I believe you have "rows" (lines of the file?) and "columns" somehow mixed up... and I see no way to use the contents of the third line without stashing away the first two somehow to process later. Or is the processing to begin with the fourth? Commented Apr 4, 2013 at 15:56
  • This question is a good place to start Commented Apr 4, 2013 at 16:19
  • Thanks, guys. I was hoping to use awk only to accomplish the task. But since awk scans the file line by line, my alternative is to simplely arrange my the criteria line/row (3rd line in my example) to be the header for awk to scan it first. I have been hesitant to take that approach since my actual file is quite large. Commented Apr 4, 2013 at 16:35
  • The other solution is a two pass filtering, as described in the question I mentioned above. Commented Apr 4, 2013 at 17:05

3 Answers 3

Reset to default
1

You could do:

code=$(
  awk '
    NR == 3 {
      for (i=1; i<=NF; i++)
        if ($i > 5) { printf "%s", sep "$" i; sep="," }
      exit sep == ""
    }' file
) &&
  awk "{print $code}" file

That is call awk twice on the same file. The first one reads the 3rd line to construct code for the second awk invocation. It exits after processing the 3rd line, so won't read the whole file fully. It outputs something like $5,$6,$7, so the next awk invocation becomes:

awk '{print $5,$6,$7}' file
Improve this answer
1
  • This is clever ! Thank you so much. Really appreciated ! Commented Apr 5, 2013 at 14:44
0

I got another awk solution to share : 

cat > extract.columns.awk   
BEGIN {   
  infil=ARGV[1]  
    while (getline < infil > 0)  
      if (++n==3)  
      {  
        for(i=1;i<=NF;i++)  
            if ($(i) > 5) x[++j]=i  
    }  
close(infil)    
}  
{  
    for (i=1;i<j;i++)  
    printf("%s ",$x[i])  
    printf("%s\n",$x[j])  
}

awk -f extract.columns.awk file

Improve this answer
3
  • You may want to exit at the end of your BEGIN statement if j == 0 to avoid processing the file if there's nothing to display. Commented Apr 5, 2013 at 15:05
  • Thank you, Stephane ! Do you mean adding " if (j==0) exit " at the end of the BEGIN block? Would you mind explaining a bit more? Commented Apr 5, 2013 at 22:22
  • Yes, that's what I meant. Commented Apr 5, 2013 at 22:24
0

Please make sure to mention the file name twice as mentioned in the below code

awk 'NR == FNR{if(FNR == line) {for(i=1; i<=NF; i++) {if($i > lmt) a[i]} close(FILENAME)} next}
{for(i=1; i<=NF; i++) {for(i in a) {out = (out == "") ? $i : (out FS $i)}}
  print out; out=""}' line=3 lmt=5 file file
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.