Bash extract lines from text file and add additional columns containing variable

Question

I have 2 variables, i and j. For each combination of the variables, there is a text file, text.txt, with text arranged in space-separated columns, e.g.:

string1 43245 7.45
string2 23452 9.34
string3 23652 2.57

I would like to do the following: Make one large text file containing all text files with the variables added in additional columns, and some other additional columns containing text, not from variables. E.g.

i = x01
j = y01

Then the output should be:

x01 y01 extrastring string1 43245 7.45
x01 y01 extrastring string2 23452 9.34
x01 y01 extrastring string3 23652 2.57

I have tried something like:

awk '{$0="'"$i"' '"$j"' extrastring"$0}'1 $i/$j/text.txt >> newtext.txt

That doesn't work.

Edit: I probably need to add that this should run in a double for loop:

for i in x01 x02 x03
do
    for j in y01 y02 y03
    do
    awk '{$0="'"$i"' '"$j"' extrastring"$0}'1 $i/$j/text.txt >> newtext.txt
    done
done

Edit: I tried this based on SoFan's answer

for i in x01 x02 x03
do
    for j in y01 y02 y03
    do
    cd $i/$j/
    awk -v i="$i" -v j="$j" '{print i,j,"extrastring",$0}' text.txt >> newtext.txt
    done
done

Unfortunately it doesn't work. The script won't run.

So is test.txt constant and you want to print it with different columns for every pair of x01.. y01..? — fedorqui
– fedorqui, Commented Aug 8, 2016 at 8:39
Sorry, no the text file is not constant. There is a different text.txt in every $i/$j/ - folder, e.g. x01/y01/text.txt, x01/y02/text.txt, etc... — Anders
– Anders, Commented Aug 8, 2016 at 8:47

P.... · Accepted Answer · 2016-08-08 08:17:43Z

1

awk -v i="x01" -v j="y01" '{print i,j,"extrastring",$0}' text.txt
x01 y01 extrastring string1 43245 7.45
x01 y01 extrastring string2 23452 9.34
x01 y01 extrastring string3 23652 2.57

Or if you have already declared variable :

i=x01
j=y01
awk -v i="$i" -v j="$j" '{print i,j,"extrastring",$0}' text.txt

answered Aug 8, 2016 at 8:17

P....

18.6k3 gold badges36 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Anders Over a year ago

This looks right, but it won't work. Can't figure out why

Some programmer dude · Accepted Answer · 2016-08-08 08:16:17Z

0

How about using the read command to read the files, and output using echo?

Something like

cat file.txt | while read line; do
    echo "$i $j extrastring $line"
done > newfile.txt

answered Aug 8, 2016 at 8:16

Some programmer dude

411k36 gold badges420 silver badges655 bronze badges

1 Comment

P.... Over a year ago

cat not really needed !?!

Mustafa DOGRU · Accepted Answer · 2016-08-08 08:37:21Z

try this;

test.sh:

#!/bin/bash
dataFile=$1
varFile=$2
line=$(awk -F = '{print $2}' $varFile | tr '\n' ' ')
awk -v variables="${line}" '{print variables "extrastring " $0 }' $dataFile


user@host:/tmp$ cat dataFile 
string1 43245 7.45
string2 23452 9.34
string3 23652 2.57

user@host:/tmp$ cat varFile 
i = x01
j = y01

user@host:/tmp$ ./test.sh dataFile varFile 
 x01  y01 extrastring string1 43245 7.45
 x01  y01 extrastring string2 23452 9.34
 x01  y01 extrastring string3 23652 2.57

James Brown · Accepted Answer · 2016-08-08 09:03:58Z

Version 3:

for var1 in x01 x02
do 
    for var2 in y01 y02
    do 
        awk '{ print "'"$var1 $var2"'" " extrastring " $0}' $var1/$var2/text.txt
    done
done
x01 y01 extrastring string1 43245 7.45
x01 y01 extrastring string2 23452 9.34
x01 y01 extrastring string3 23652 2.57
x01 y02 extrastring string1 43245 7.45
x01 y02 extrastring string2 23452 9.34
x01 y02 extrastring string3 23652 2.57
x02 y01 extrastring string1 43245 7.45
x02 y01 extrastring string2 23452 9.34
x02 y01 extrastring string3 23652 2.57
x02 y02 extrastring string1 43245 7.45
x02 y02 extrastring string2 23452 9.34
x02 y02 extrastring string3 23652 2.57

Didn't test it under directory structure, though.

Collectives™ on Stack Overflow

Bash extract lines from text file and add additional columns containing variable

4 Answers 4

1 Comment

1 Comment

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

1 Comment

Comments

Comments

Related