How to replace "&" with "&" except when already part of an XML entity?

Question

I have a file (xml) with many &/</>/etc, but sometimes also some & characters by themselves. I want to change this symbol to & without replacing the others.

cat file.xml | sed s/"&"/"&"/g | sed s/"&"/"&"/g > new_file.xml

This does not work, because it also replaces the &s found in &, > for example.

How to do it?

Answer 1

You could first escape the & when found in an entity, and then substitute the remaining ones. Like:

LC_ALL=C sed 's/_/_u/g; # use _ as an escape character. Here escape itself
              s/&\([[:alpha:]][[:alnum:]]*;\)/_a\1/g; # replace & with _a when in entities
              s/&\(#[0-9]\{1,8\};\)/_a\1/g; # Ӓ case
              s/&\(#x[0-9a-fA-F]\{1,8\};\)/_a\1/g; # ꯍ case
              s/&/\&/g; # now convert the non-escaped &s
              s/_a/\&/g;s/_u/_/g; # restore escaped & and _'

With perl:

perl -pe 's/&(?!#?\w{1,31};)/&/g'

That one is a bit more lax than the sed one in that it would consider as XML entity anything that starts with &, an optional # and any number (up to 31) of alnums (or underscores) and ;, while the sed one would be more explicit in what an entity is (as in &#blah; would not be considered as an entity). In practice, it's not likely to make a lot of difference.