Using a reference to a bash string variable in sed

Question

I'm a relative Linux novice. Suppose that I have a text file a.txt that contains the following text:

A
B
C

Now, if I want to change the text on line 2 (which contains B), I can use the following sed command:

sed -i '2s/^.*$/X/' a.txt

to change the B to X, for example.

But now suppose that I want to write an sed command that changes the text on line 2 to something that includes a bash string variable. I want to change the text on line 2 to fname="myaddress", where the quotation marks are included explicitly, and where myaddress is a bash string variable (defined somewhere else). To try to do this, I have created a bash shell script called mytest.sh and containing the following:

#!/bin/bash

myaddress="/home/"
echo fname=\"$myaddress\"

head a.txt
sed -i '2s/^.*$/fname=\"$myaddress\"/' a.txt
head a.txt

Now, echo fname=\"$myaddress\" prints fname="/home/", which is the text that I want to be on line 2 of a.txt. However, sed instead prints fname="$myaddress" to line 2 of a.txt, probably because it is inside the surrounding apostrophes. In other words, the output of the above bash shell script is:

fname="/home/"
A
B
C

A
fname="$myaddress"
C

My question is, how can I get sed to actually print the value stored in myaddress and referenced by $myaddress?

Answer 1

If you want the replace to work whatever the content of the variable is, you have to escape characters in the variable that are special on the right hand side of a s command in sed which are:

• the delimiter itself (above, you used /)
• & which is replaced by the matched string
• newline which needs to be escaped
• \ which is used to escape all of the above and introduce \1, \2...

Like:

escaped_var=$(printf '%s\n' "$var" | sed 's:[/&\]:\\&:g;s/$/\\/')
escaped_var=${escaped_var%?}

sed "s/regexp/$escaped_var/g"

As that's quite cumbersome, you might want to use perl instead:

v=$var perl -pe 's/regexp/$ENV{v}/g'

The equivalent of sed's 2s would be:

v=$var perl -pe 's/regexp/$ENV{v}/g if $. == 2'

Though in your case of course, you'd rather write it:

v=$myaddress perl -pe '$_ = "fname=\"$ENV{v}\"\n" if $. == 2'

The other advantage is that it works with binary data as well and you can use a -i option for in-place editing while with sed, you could only do that on systems where sed is the GNU sed.

Also note that the NUL character cannot be stored in an environment variable.

Answer 2

The shell doesn't interpret variables within single quotes. You need to change them to double quotes. Since your variable has slashes in it, you should use a different delimiter:

sed -i "2s|^.*\$|fname=\"$myaddress\"|" a.txt

Note that I escaped the end-of-line $ so it isn't interpreted by the shell. In this case it wouldn't be, but you should get in the habit of escaping $s you want to keep in double quotes. You could also do this using a couple pairs of quotes:

sed -i '2s|^.*$|fname="'"$myaddress"'"|' a.txt

Answer 3

There are two problems: First, as already noted by Kevin, shell variables need to be enclosed in double quotes if they are to be expanded by the shell. Second, your replacement string contains a /, so you cannot use / at the same time as a delimiter for sed. If you know that some character, say ,, will definitely not occur in the replacement string, you could use that character as sed delimiter, so you get

sed -i '2s,^.*$,fname="'"$myaddress"'",' a.txt