Use sed to replace a string within a range using a multi-line text range

Question

I have the following in a file (values.yaml):

global:
  repo1:
    enabled: true
  repo2:
    enabled: true

repo1:
  replicaCount: 1
  image:
    tag: latest
    pullPolicy: Always

repo2:
  replicaCount: 1
  image:
    tag: latest
    pullPolicy: Always

and I want to replace the tag: latest for repo1 with something like tag: newest1 and tag: latest for repo2 with something like tag: newest2 so that I end up with:

global:
  repo1:
    enabled: true
  repo2:
    enabled: true

repo1:
  replicaCount: 1
  image:
    tag: newest1
    pullPolicy: Always

repo2:
  replicaCount: 1
  image:
    tag: newest2
    pullPolicy: Always

So I'm trying to search a range from repo1:\n replicaCount:1 to either pullPolicy or the end of the file so I get a range that has just the one tag in it and can replace it.

I have

sed -i "" "N;/repo1:\n  replicaCount:/,/pullPolicy/s/tag:.*/tag: newest1/g" values.yaml

and that almost works, but it always deletes the very last pullPolicy line like:

global:
  repo1:
    enabled: true
  repo2:
    enabled: true

repo1:
  replicaCount: 1
  image:
    tag: newest1
    pullPolicy: Always

repo2:
  replicaCount: 1
  image:
    tag: newest2

And I'm on a Mac, which is why all the double quotes.

How do I specify the end range to either be the end of the file (I've also tried $ to no avail) or to be the pullPolicy?

sed would be a poor choice of tool for doing this. See mikefarah.gitbook.io/yq for how to install/use a yaml editor and if you can't install yq for some reason then you'd use awk. — Ed Morton
– Ed Morton, Commented Aug 25, 2020 at 20:31
Change the N in your sed code to $!N. The lonely N is causing the last line to not be printed. Take care to backslash it since you are inside double quotes. For a robust way lookat my solution using python. I like your approach of combining N with range. — Rakesh Sharma
– Rakesh Sharma, Commented Aug 26, 2020 at 4:30

ewen-goisot · Accepted Answer · 2020-08-26 17:35:26Z

sed -z uses null character separator instead of newlines.

cat values.yaml | sed -z "s/\nrepo\([0-9]*\):\n\(\([^\n]\+\n\)*\)\([ \t]*\)tag:[^\n]*/\nrepo\1:\n\2\4tag: new\1/g"

does what you want.

\nrepo$[0-9]*$:\n matches with a line starting with repo, and records its value in \1.
$\([^\n]\+\n$*\) matches with any number of non-empty lines (\+ is for at least one char), and records its value in \2. \3 is wasted for the inside group.
$[ \t]*$tag:[^\n]* matches with a line containing the word tag with space indent.
\nrepo\1:\n\2\4 lets the first lines unchanged.
tag: new\1 changes the tag value, with new + the repo value.
/g is used to do both changes with a single command.

If it does not work, you can try with gnu-sed (can be installed on MacOS too) instead of your default sed.

nezabudka · Accepted Answer · 2020-08-26 18:38:50Z

1

sed -ri ':1;/^repo[12]/,/tag/!b;/tag/!{N;b1};s/^(repo(.).*)latest$/\1newest\2/'

Do not forget about the l command for debugging, which shows what you have in the current buffer. e.g:

sed -nr ':1;/^repo[12]/,/tag/!b;/tag/!{N;b1};l;s/^(repo(.).*)latest$/\1newest\2/' file

repo1:\n  replicaCount: 1\n  image:\n    tag: latest$
repo2:\n  replicaCount: 1\n  image:\n    tag: latest$

Production:

sed -ri '/^repo[12]/!b;:1;/tag/!{N;b1};s/^(repo(.).*)latest$/\1newest\2/' file

edited Aug 26, 2020 at 18:38

answered Aug 26, 2020 at 18:19

nezabudka

2,4567 silver badges15 bronze badges

Add a comment |

Rakesh Sharma · Accepted Answer · 2020-08-26 02:51:11Z

Using python the way to edit this file is to import the yaml module then load the yaml file in a python dictionary, aka, hash | associative array, albeit a nested one.

Then we go in n tweak the values of the keys in the yaml file.

Finally we save the dictionary back into the yaml file.

The sed code is to insert empty lines on the top level nodes if yaml file.

$ yaml_file='values.yaml'

$ python3 - "$yaml_file" <<\YAML
import sys, yaml

yaml_doc = sys.argv[1]

with open(yaml_doc) as fp:
  h = yaml.safe_load(fp)

h['repo1']['image']['tag'] = 'newest1'
h['repo2']['image']['tag'] = 'newest2'

with open(yaml_doc, 'w') as fp:
  yaml.safe_dump(h, fp, default_flow_style=False)
YAML

$ sed -Ei -e '1!{/^\S/{H;z;x;};}' "$yaml_file"

Stack Exchange Network

Use sed to replace a string within a range using a multi-line text range

3 Answers 3

You must log in to answer this question.

Hot Network Questions

Use sed to replace a string within a range using a multi-line text range

3 Answers 3

You must log in to answer this question.

Related

Hot Network Questions