linux shell script to remove 1 char in a particular field in file having lines of around 3000

Question

• My input file:

  1oo+457864227yexaloo+6784536pkp8907654
  2oo+499004227yexaloo+69008908pkp8907654
  3oo+648968976yexaloo+53589094pkp8907654
  4oo+490764578yexaloo+6784536pkp8907654
  

I want to find out the length of digits between second occurance of oo+ and pkp and if length is greater than 7 digits, then I want to strip of first char o in oo of second occurance only

• Desired output file

  1oo:457864227yexaloo:6784536pkp8907654
  2oo:499004227yexalo:69008908pkp8907654
  3oo:648968976yexalo:53589094pkp8907654
  4oo:490764578yexaloo:6784536pkp8907654
  

Can someone help, how to achieve this?

Answer 1

This perl one-liner will find the second occurrence of oo+ and then capture the longest string of digits after that until the first pkp. Note that it assumes only digits between the second oo+ and the pkp. Then, depending on the length of the stretch of digits, it will print either o+ or oo+. This is what your question seems to be asking for:

$ perl -nle '/(.*oo\+.*?)oo\+((\d+)pkp.*)/; 
             print length($3) > 7 ? "${1}o+$2" : "${1}oo+$2"' file
1oo+457864227yexaloo+6784536pkp8907654
2oo+499004227yexalo+69008908pkp8907654
3oo+648968976yexalo+53589094pkp8907654
4oo+490764578yexaloo+6784536pkp8907654

However, the output you show isn't actually what your question describes. You have also changed the + after each of the two oo to a :. If you need that as well, try

$ perl -nle 's/oo\+/oo:/g; /(.*oo:.*?)oo:((\d+)pkp.*)/; 
             print length($3) > 7 ? "${1}o:$2" : "${1}oo:$2"' file
1oo:457864227yexaloo:6784536pkp8907654
2oo:499004227yexalo:69008908pkp8907654
3oo:648968976yexalo:53589094pkp8907654
4oo:490764578yexaloo:6784536pkp8907654

Explanation

• perl -nle : The -n tells perl to open ints input file and then apply the script given by -e to each line. The -l removes trailing newlines from the input and adds a newline to each print call.
• s/oo\+/oo:/g;: replace all occurrences of oo+ with oo:.
• /(.*oo:.*?)oo:((\d+)pkp.*)/;: the parentheses are "capture groups"; anything matched using one of the patterns in the parentheses is then going to be available as $N, with the first captured group being $1, the second $2 and so on. Here, $1 will be everything until the second oo: (not oo+ because we have replaced the + with : in the previous step), $2 will be everything after the second 00: until the end of the line and $3 will be the numbers you are looking for (since the parentheses are nested, this works as ( external paren is $2 (internal is $3))).
• print length($3) > 7 ? "${1}o:$2" : "${1}oo:$2" : this is a shorthand that could be written as "if the length of $3 (the numbers) is more than 7, print $1 then o then : and $2, and if the length is less than 7 print $1 then oo then : and $2. The general syntax is command test ? case1 : case2 which is read as if(test){ command case1} else{ command case2 }.

Answer 2

Another perl approach:

$ perl -pe 's/oo\+.*o\Ko(?=\+\d{8,}pkp)//' your-file
1oo+457864227yexaloo+6784536pkp8907654
2oo+499004227yexalo+69008908pkp8907654
3oo+648968976yexalo+53589094pkp8907654
4oo+490764578yexaloo+6784536pkp8907654
$ perl -pe 's/oo\+.*o\Ko(?=\+\d{8,}pkp)//; y/+/:/' your-file
1oo:457864227yexaloo:6784536pkp8907654
2oo:499004227yexalo:69008908pkp8907654
3oo:648968976yexalo:53589094pkp8907654
4oo:490764578yexaloo:6784536pkp8907654

Answer 3

Using any awk:

$ awk -F'oo[+]' '{print $1 "oo:" $2 ( index($3,"pkp") > 8 ? "o:" : "oo:" ) $3}' file
1oo:457864227yexaloo:6784536pkp8907654
2oo:499004227yexalo:69008908pkp8907654
3oo:648968976yexalo:53589094pkp8907654
4oo:490764578yexaloo:6784536pkp8907654

Answer 4

It is rather simple with awk (requires an Awk version with gensub(), such as GNU Awk):

$ awk '/oo\+[[:digit:]]{8,}pkp/{$0=gensub(/oo\+/,"o+",2)}1' input.txt

1oo+457864227yexaloo+6784536pkp8907654
2oo+499004227yexalo+69008908pkp8907654
3oo+648968976yexalo+53589094pkp8907654
4oo+490764578yexaloo+6784536pkp8907654

• This will test if the line contains the pattern "oo+, followed by eight or more digits, followed by pkp".
• If so, the gensub() function is used to replace the second match of oo+ by o+ on the current line ($0).
• The seemingly stray 1 will the print the current line including any modifications.

---

If you actually want to additionally change the o+ to o:, you need to modify the program as follows:

$ awk '{gsub(/oo\+/,"oo:")} /oo:[[:digit:]]{8,}pkp/{$0=gensub(/oo:/,"o:",2)}1' input.txt

1oo:457864227yexaloo:6784536pkp8907654
2oo:499004227yexalo:69008908pkp8907654
3oo:648968976yexalo:53589094pkp8907654
4oo:490764578yexaloo:6784536pkp8907654

This will first replace all occurences of oo+ with oo: and then apply the same logic as above for oo:,8 or more digits,pkp.

Note that all of the above relies on the structure you presented, i.e. only the second oo+ is followed by digits and then pkp. If the first oo+ can also be followed by only digits and then pkp, or the second happens to not be, but a possibly further occurence of oo+ is, it will break down.

Answer 5

With standard sed, test for lines that contain oo+ followed by eight digits and the substring pkp. Change the second occurrence of oo+ into o+ on the matching lines when found.

$ sed '/oo+[0-9]\{8\}pkp/ s/oo+/o+/2' file
1oo+457864227yexaloo+6784536pkp8907654
2oo+499004227yexalo+69008908pkp8907654
3oo+648968976yexalo+53589094pkp8907654
4oo+490764578yexaloo+6784536pkp8907654

To also change all occurrences of + into :, use y/+/:/:

$ sed -e '/oo+[0-9]\{8\}pkp/ s/oo+/o+/2' -e 'y/+/:/' file
1oo:457864227yexaloo:6784536pkp8907654
2oo:499004227yexalo:69008908pkp8907654
3oo:648968976yexalo:53589094pkp8907654
4oo:490764578yexaloo:6784536pkp8907654

If you need to ensure that there are two sets of oo+ on the lines and that the eight digits occur between the second set and the pkp string, change the regular expression oo+[0-9]\{8\}pkp into oo+.*oo+[0-9]\{8\}pkp.

---

Another approach is to interpret the input as a set of +-delimited fields and to truncate the 2nd such field by one character whenever the 3rd field starts with eight digits.

Here I'm using Miller (mlr) to do the operation and also to change the field delimiters to colons:

$ mlr --nidx --ifs '+' --ofs ':' put '$3 =~ "^[0-9]{8}" { $2 = truncate($2,strlen($2)-1) }' file
1oo:457864227yexaloo:6784536pkp8907654
2oo:499004227yexalo:69008908pkp8907654
3oo:648968976yexalo:53589094pkp8907654
4oo:490764578yexaloo:6784536pkp8907654

Answer 6

Using gawk:

$ awk '
     n=split($0,a, "oo\\+",sep){ 
        for(i in sep) sep[i]="oo:";
        if (match(a[3], /^.{8,}pkp/)) sep[2]="o:";
        for(i=1;i<=n;i++) $0 = (i==1 ? "" : $0) sep[i-1] a[i]
  }1'

$ awk -F 'oo\\+' -v OFS='oo:' '{for(i=1;i<=NF;i++) printf "%s", $i (i==NF ? ORS : (i==2  && match($3, /^.{8,}pkp/) ? "o:" : OFS))}' 

Answer 7

Using Raku (formerly known as Perl_6)

~$ raku -pe 's/ oo\+ .* o <( o <?before \+ \d**8..* pkp > //;'  file

#OR:

~$ raku -pe 's:2nd/ <( o )> o \+ \d**8..* //;'  file  

• The first answer uses Raku's <?before … > positive-lookahead in conjunction with a <( capture marker. This answer is inspired by the Perl answer by @StéphaneChazelas.

• The second answer uses a pair of capture markers, <( … )> in conjunction with Raku's :2nd named parameter (i.e. adverb). We see here that the capture markers allow the entire pattern to be recognized, after which the second match is chosen and the capture markers take effect (to only delete an o). This answer assumes that the first oo+ pattern is immediately followed by at least 8 digits.

Sample Input:

1oo+457864227yexaloo+6784536pkp8907654
2oo+499004227yexaloo+69008908pkp8907654
3oo+648968976yexaloo+53589094pkp8907654
4oo+490764578yexaloo+6784536pkp8907654

Sample Output:

1oo+457864227yexaloo+6784536pkp8907654
2oo+499004227yexalo+69008908pkp8907654
3oo+648968976yexalo+53589094pkp8907654
4oo+490764578yexaloo+6784536pkp8907654

https://docs.raku.org
https://raku.org

Answer 8

#!/bin/bash

user_input="$1"

# Read input into an array
mapfile -t strings_file < "$user_input"

# Iterate and extract numbers
for string in "${strings_file[@]}"; do
    number_extract=$(echo "$string" | sed -n 's/.*oo+.*oo+\(.*\)pkp.*/\1/p')

    count_digits=${#number_extract}

    # Check if the length is greater than 7
    if [ "$count_digits" -gt 7 ]; then
        string=$(echo "$string" | sed 's/oo/o/2')
    fi

    # Replace + with :
    string=$(echo "$string" | sed 's/+/:/g')

    echo "$string" >> output_"$user_input"
done

exit 0

Save to: char_edit.sh
Make executable: chmod +x char_edit.sh
Run with: ./char_edit.sh your_file