Search and replace lines AFTER a regex match using "sed"

Question

This kind of feels like it would be easier in awk, but I'm curious if sed can do it. Here is my input:

line 1
line 2
line 3
line 1
line 2
line 3
line 1
line 2
line 3

I'd like to write an in-place regex that finds the second line 1, then replaces all line 3's found after that. The output would look like this:

line 1
line 2
line 3
line 1
line 2
replaced
line 1
line 2
replaced

I'm not really looking for "clever" solutions that only apply to this input. I want to learn if there is a general-purpose way to search and replace after a match with sed.

I thought the solution would be somewhere in the addr documentation, but it doesn't seem to describe /starting point/,s/... as something you can do, and I'm getting error when I attempt to do so.

Answer 1

To do this, you need to count the number of times you've already seen the target line. Unfortunately, counting anything in sed is a PITA. You'd be better off doing this in awk or perl. For example:

$ perl -p -e '$found++ if m/line 1/;
              next if $found < 2;
              s/line 3/replaced/' input.txt 
line 1
line 2
line 3
line 1
line 2
replaced
line 1
line 2
replaced

or in awk:

$ awk '/line 1/ { found++ };
       found < 2 { print ; next };
       { sub(/line 3/,"replaced") ; print }' input.txt 
line 1
line 2
line 3
line 1
line 2
replaced
line 1
line 2
replaced

These work, but are far from optimal solutions, especially the awk version (which is just a direct translation of the perl).

BTW, it wouldn't be difficult at all to make either version take the regex to search for and count, the regex to search for and replace, the replacement string, and even the required count as command-line options instead of hard-coding them in the script.

Answer 2

With GNU sed (for the 0 address¹):

$ sed '0,/^line 1$/b; /^line 1$/,$ s/^line 3$/replaced/' < file
line 1
line 2
line 3
line 1
line 2
replaced
line 1
line 2
replaced

We branch out for the first 0,/^line 1/ range, which means the second /^line 1$/,$ only sees its starting line when reaching the second occurrence of line 1.

With awk, it's easier to spot the n^th occurrence of line 1:

awk '$0 == "line 1" && ++n == 2, eof {if ($0 == "line 3") $0 = "replaced"}; 1'

(here eof is just any other uninitialised variable, so which yields false to signify that that first, last range has no end; same as just using 0 literally though more legible; similarly, we could use !skip instead of 1 to signify true for the current record to be printed but using 1 is idiomatic enough that it should be recognised by all awk users and a more legible alternative would rather be {print}).

---

^{¹ and since we assume GNU sed, we can also add ;more commands after the b one, which is not standard either. We'd need another -e 'more commands' with other seds.}

Answer 3

Create a counter in the buffer (pattern space) and check it on each line:

sed '
/^line 1/{x;/11/!s/^/1/;x}
x;/11/{x;b1};x;b
:1;s/^line 3$/replaced/
' file

we use 1 as the counter unit

[update] Better performance:

sed '
/^line 1$/{x;/11/!s/^/1/;x}
/^line 3$/!b
x;/11/!{x;b}
x;s/.*/replaced/
' file

The counter is checked only in the lines matching the pattern - /^line 3$/

Answer 4

Using GNU sed specifically:

sed -e '0,/^line 1$/ b' \
    -e '//,$ s/^line 3$/replaced/' file

This first outputs all lines between the start of the file and the first line 1 line. Using 0 as the start of an address range like this requires GNU sed and enables us to cope with the case where line 1 is the very first line of the file.

After reaching the first line 1 line and outputting it, it applies a substitution to all lines from the next line 1 line to the end of the file, replacing the line 3 lines with the text replaced.

If you want to insert multiple lines of text, use c (change) rather than s (substitute):

sed -e '0,/^line 1$/ b' \
    -e '//,$ {' \
      -e '/^line 3$/ c\' \
      -e 'replace 1\' \
      -e 'replace 2' \
    -e '}' file

Each line, apart from the last that you want to replace the line 3 lines with, must end with a backslash.

It may be easier to see what that sed script is doing if we write the script as a separate editing script:

0,/^line 1$/ b

//,$ {
        /^line 3$/ c\
replaced 1\
replaced 2
}

---

Using ed, we could find the first line 1, then perform a c (change) command on each line matching line 3 from the next line matching line 1 on to the end.

/^line 1$/; //,$ g/^line 3$/ c\
replaced

From the command line, sending the result to the terminal,

ed -s file <<'END_ED'
/^line 1$/; //,$ g/^line 3$/ c\
replaced
,p
Q
END_ED

or,

printf '%s\n' '/^line 1$/; //,$ g/^line 3$/ c\' 'replaced' ',p' 'Q' |
ed -s file

To replace the line 3 lines with multiple lines of text, make sure that each line of text added is terminated by a backslash:

ed -s file <<'END_ED'
/^line 1$/; //,$ g/^line 3$/ c\
replaced\
more replace text\
even more
,p
Q
END_ED

To do an in-place edit, use w (write the buffer to file) and q (quit) in place of ,p (print all lines in the buffer) and Q (quit, unconditionally) in the editing script.

Note that ed is unsuitable for editing huge files, as the editor reads the file into memory.

---

If you only want to replace the lines with a single line of text, you could avoid a multi-line ed script by using s (substitute) in place of c (well, you need ,p and Q too):

/^line 1$/; //,$ g/^line 3$/ s//replaced/

Testing:

$ printf '%s\n' '/^line 1$/; //,$ g/^line 3$/ s//replaced/' ',p' 'Q' | ed -s file
line 1
line 2
line 3
line 1
line 2
replaced
line 1
line 2
replaced

Answer 5

Using GNU sed we can perform the task by storing the count as dots . in the hold space.

sed -e '
  /^line 1$/,/^line 1$/{//!b
    x;s/^$/./;x;t
    :a;n;s/^line 3$/REPLACED/;ba
  }
' file

Another possibility without using the range operator and with extended regex mode in GNU sed. Here the newline \n function as counter, stored in the hold space.

sed -Ee '
  /^line 1$/H
  x;s/^(\n{1,2}).*/\1/;x
  /^line 3$/G
  /\n.{2}/c REPLACED
  s/\n//g
' file

---

Perl can do the same in a similar fashion

perl -lpe '
  s/^line 3$/$a?"REPLACED":$&/e
           unless
  m?^line 1$? ... /^line 1$(?{$a++})/;
' file

---

Using awk, we can nicely combine regexes with variables and incrementing by one

awk '
1 < ( k += /^line 1$/ ) &&
sub(/^line 3$/,"REPLACED") ||
1' file