Extracting Text from line

Question

I need help extracting certain part of line from file.

Here is how my file looks like:

testfile.txt
This is a test line 1 $#%#
This is a test line 2 $#%#
This is a test line 3 $#%#
This is a test line 4 $#%#
This is a test line 5 $#%#
This is a test line 6 $#%#
This is a test line 7 $#%#

and here is my bash script:

#!/bin/bash

while read line
do
#echo $line
FilterString=${line:22:26}
echo $FilterString>>testfile2.txt
done<testfile.txt

The above script gets the string $#%# and writes to temp file

My Question:

Instead of writing the string $#%# i want everything except string $#%# written to file. So i want my final output file to look like:

testfile.txt
This is a test line 1 
This is a test line 2 
This is a test line 3 
This is a test line 4 
This is a test line 5 
This is a test line 6 
This is a test line 7

Please also suggest me the best tool for doing it

Thanks in advance.

fedorqui · Accepted Answer · 2013-10-28 09:36:31Z

If it is just the last field you want to delete, you can use awk:

$ awk 'NF=NF-1' file
This is a test line 1
This is a test line 2
This is a test line 3
This is a test line 4
This is a test line 5
This is a test line 6
This is a test line 7

It decreases in one the number of fields, so that the last one is not taken into consideration.

Then, it performs the default action of awk that is {print $0}.

To redirect to a file, use awk 'NF=NF-1' file > new_file.

Update

As per your comment

In my case it will not always be the last field, it might also be in between the other fields however at predefined position(always fixed position).

Then you can use the following awk syntax:

awk -v c=col_num '{$(c)=""}1' file

where col_num can be set manually like:

$ awk -v c=3 '{$(c)=""}1' file
This is  test line 1 $#%#
This is  test line 2 $#%#
This is  test line 3 $#%#
This is  test line 4 $#%#
This is  test line 5 $#%#
This is  test line 6 $#%#
This is  test line 7 $#%#
$ awk -v c=5 '{$(c)=""}1' file
This is a test  1 $#%#
This is a test  2 $#%#
This is a test  3 $#%#
This is a test  4 $#%#
This is a test  5 $#%#
This is a test  6 $#%#
This is a test  7 $#%#

You can also use cut like this, omitting the field you want to skip:

$ cut -d' ' -f1,2,3,4,5,6 file
This is a test line 1
This is a test line 2
This is a test line 3
This is a test line 4
This is a test line 5
This is a test line 6
This is a test line 7

$ cut -d' ' -f1,2,3,5,6,7 file
This is a line 1 $#%#
This is a line 2 $#%#
This is a line 3 $#%#
This is a line 4 $#%#
This is a line 5 $#%#
This is a line 6 $#%#
This is a line 7 $#%#

In my case it will not always be the last field, it might also be in between the other fields however at predefined position(always fixed position)
OK @Rockwire then you can play with column name. In 5 min I will update, something like awk -vcol=$col '$(col)=""' file, being $col a bash var, should make it.

Community · Accepted Answer · 2017-05-23 10:25:48Z

By saying:

FilterString=${line:22:26}

you chose to print the $#%# parts of the line.

You could have said:

FilterString=${line:0:21}

to print the desired portion of the line. Alternatively, you could say:

FilterString=${line//\$#%#/}

(note that the $ sign needs to be escaped)

Using sed, you could have said:

sed 's/ $#.*//g' testfile.txt

Supplying the -i option to sed would make the changes in-place:

sed -i 's/ $#.*//g' testfile.txt

As per your comment, if you want to remove text from fixed positions in the file, using cut might simplify things. Saying:

cut -b1-21,27- testfile.txt

would remove bytes 22-26 (inclusive) from all lines in the file testfile.txt.

anubhava · Accepted Answer · 2013-10-28 09:30:59Z

1

Instead of writing the string "$#%#" i want everything except string "$#%#" written to file.

It can be done using sed inline:

sed -i.bak 's/ *\$#%#//g' testfile.txt

answered Oct 28, 2013 at 9:30

anubhava

790k67 gold badges603 silver badges671 bronze badges

Comments

cforbish · Accepted Answer · 2013-10-28 09:40:09Z

1

You were very close:

FilterString=${line:0:22}

Or to just filter the garbage:

FilterString=${line% \$#%#}

answered Oct 28, 2013 at 9:40

cforbish

8,8893 gold badges32 silver badges32 bronze badges

Comments

Michael Kazarian · Accepted Answer · 2013-10-28 09:44:39Z

1

Try it:

#!/bin/sh

while read line
do
#echo $line
FilterString=`python -c "s='$line';print s[:s.find('$')]"`
echo $FilterString>>testfile2.txt`

This sample can work with various lenth. For example with file context:

...
This is a test line 6 $#%#
This is a test line 1024 $#%#
...

you will get next result:

This is a test line 6
This is a test line 1024

answered Oct 28, 2013 at 9:44

Michael Kazarian

4,4921 gold badge23 silver badges27 bronze badges

Comments

Rockwire · Accepted Answer · 2013-10-28 10:19:20Z

Appreciate all your answers guys:

Will use script based on @devnull's answer:

#!/bin/bash
while read line
do
#echo $line
#FilterString=${line:22:26}
echo $line | cut -b1-20,27- >>testfile2.txt
done<testfile

therefore if the file look like

testfile.txt
This is a test line 1 $#%# more text
This is a test line 2 $#%# more text
This is a test line 3 $#%# more text
This is a test line 4 $#%# more text
This is a test line 5 $#%# more text
This is a test line 6 $#%# more text
This is a test line 7 $#%# more text

then output will be:

testfile2.txt
This is a test line  more text
This is a test line  more text
This is a test line  more text
This is a test line  more text
This is a test line  more text
This is a test line  more text
This is a test line  more text

which is exactly what i wanted

Collectives™ on Stack Overflow

Extracting Text from line

6 Answers 6

Update

2 Comments

Comments

Comments

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

Update

2 Comments

Comments

Comments

Comments

Comments

Comments

Related