extracting values from text file using awk

Question

I have 100 text files which look like this:

File title
4
Realization number
variable 2 name
variable 3 name
variable 4 name
1   3452  4538   325.5

The first number on the 7th line (1) is the realization number, which SHOULD relate to the file name. i.e. The first file is called file1.txt and has realization number 1 (as shown above). The second file is called file2.txt and should have realization number 2 on the 7th line. file3.txt should have realization number 3 on the 7th line, and so on...

Unfortunately every file has realization=1, where they should be incremented according to the file name.

I want to extract variables 2, 3 and 4 from the 7th line (3452, 4538 and 325.5) in each of the files and append them to a summary file called summary.txt.

I know how to extract the information from 1 file:

awk 'NR==7,NR==7{print $2, $3, $4}' file1.txt

Which, correctly gives me:

3452 4538 325.5

My first problem is that this command doesn't seem to give the same results when run from a bash script on multiple files.

#!/bin/bash    
for ((i=1;i<=100;i++));do
    awk 'NR=7,NR==7{print $2, $3, $4}' File$((i)).txt
done

I get multiple lines being printed to the screen when I use the above script.

Secondly, I would like to output those values to the summary file along with the CORRECT preceeding realization number. i.e. I want a file that looks like this:

1  3452  4538  325.5
2  4582  6853  158.2
...
100  4865 3589  15.15

Thanks for any help!

Jonathan Leffler · Accepted Answer · 2012-12-12 00:28:02Z

3

You can simplify some things and get the result you're after:

#!/bin/bash    

for ((i=1;i<=100;i++))
do
    echo $i $(awk 'NR==7{print $2, $3, $4}' File$i.txt)
done

You really don't want to assign to NR=7 (as you did) and you don't need to repeat the NR==7,NR==7 either. You also really don't need the $((i)) notation when $i is sufficient.

If all the files are exactly 7 lines long, you can do it all in one awk command (instead of 100 of them):

awk 'NR%7==0 { print ++i, $2, $3, $4}' Files*.txt

answered Dec 12, 2012 at 0:28

Jonathan Leffler

759k145 gold badges961 silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Flux Capacitor Over a year ago

Thanks. The first bit actually didn't work for me. I was getting substitution errors or syntax errors or something. However, the second 1 line command worked well for me. Thanks!

Jonathan Leffler Over a year ago

That's odd; the only way I could see running into problems is if you didn't actually have 100 files. I tried it with 10 files and it seemed to work fine with both Bash 3.2 and 4.2. However, I'm glad the one-liner worked; it is neater anyway.

P.P · Accepted Answer · 2012-12-12 00:27:55Z

Notice that you have only one = in your bash script. Does all the files have exactly 7 lines? If you are only interested in the 7th line then:

#!/bin/bash    
for ((i=1;i<=100;i++));do
    awk 'NR==7{print $2, $3, $4}' File$((i)).txt
done

Since your realization number starts from 1, you can simply add that using nl command.

For example, if your bash script is called s.sh then:

./s.sh | nl > summary.txt

will get you the result with the expected lines in summary.txt

Steve · Accepted Answer · 2012-12-12 00:51:49Z

0

Here's one way using awk:

awk 'FNR==7 { print ++i, $2, $3, $4 > "summary.txt" }' $(ls -v file*)

The -v flag simply sorts the glob by version numbers. If your version of ls doesn't support this flag, try: ls file* | sort -V instead.

answered Dec 12, 2012 at 0:51

Steve

55.1k13 gold badges94 silver badges105 bronze badges

Collectives™ on Stack Overflow

extracting values from text file using awk

3 Answers 3

2 Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Related