Evaluate captured expression inside replacement in sed

Question

I'm working on deployment script and my need is to replace database (or another system) credentials in configuration file.

I have PHP files with credentials like:

<?php

return [
    'mysql-rw' => [
      'dns' => 'mysql-rw.local',
      'user' => 'local',
      'password' => 'PASS__LABELONE__',
    ],
    'mysql-ro' => [
      'dns' => 'mysql-ro.local',
      'user' => 'local',
      'password' => 'PASS__LABELTWO__',
    ],
];

Also I have Bash environment variables in CI server - LABELONE and LABELTWO. I want to replace templates in file with one-line sed command, but have error or wrong substitution:

LABELONE=pass1 LABELTWO=pass2 sed -r "s/PASS__(.+)__/${!\1}/" tst.txt 
bash: s/PASS__(.+)__/${!\1}/: bad substitution

My approach is:

1. Find template in file
2. Capture a label from it (in PASS__LBL__ it is LBL)
3. Replace template with env. variable ($LBL)

How can I evaluate such dynamic variables in sed or it is unreal?

Answer 1

Perl can access env variables through the %ENV hash:

LABELONE=pass1 LABELTWO=pass2 perl -pe 's/PASS__(.+)__/$ENV{$1}/' -- tst.txt

Answer 2

The value of captured group (...) from sed expresion can't be processed as dynamic bash environment variable, use GNU awk program instead:

export LABELONE="pass1"
export LABELTWO="pass2"

awk 'match($0, /PASS__([^_]+)__/, m){ sub(/PASS__[^_]+__/, ENVIRON[m[1]]) }1' testfile

The output:

<?php

return [
    'mysql-rw' => [
      'dns' => 'mysql-rw.local',
      'user' => 'local',
      'password' => 'pass1',
    ],
    'mysql-ro' => [
      'dns' => 'mysql-ro.local',
      'user' => 'local',
      'password' => 'pass2',
    ],
];

---

ENVIRON
An associative array containing the values of the environment. The array indices are the environment variable names; the elements are the values of the particular environment variables.

Answer 3

Probably not what you are looking for, but if you sequentially pipe it through multiple seds with explicit matching of the labels it does work... Not elegant of course.

LABELONE=pass1; LABELTWO=pass2;
sed -r "s/PASS__LABELONE__/$LABELONE/" tst.txt | sed -r "s/PASS__LABELTWO__/$LABELTWO/"

Of course this will get tedious real quick if you have more variables or if they are named something else. You could probably do something in bash itself but I myself can't see how to do it in one line of sed unfortunately.

EDIT: Ok so this works, assuming that the variables are indeed environment variables and you have access to envsubst

export LABLEONE=pass1

export LABELTWO=pass2

sed -r "s/PASS__(.+)__/$\1/g" test.txt | envsubst

It relies on envsubst however, which is part of the gettext I think. So don't know whether that would work for you? Also, by writing the actual $LABELONE variables in the configuration files you can get rid of the sed altogether and just use envsubst. Personally I like the perl script mentioned in another comment so I voted for that.