Unexpected in-place sed behavior

Question

In trying to write a sed one-liner to replace instances of either

infra/helm/*/charts/*

or

infra/helm/*/charts/

In a file with "infra/helm/*/charts/*", a newline, and another line of text looking like so

infra/helm/*/charts/*
infra/helm/*/manifests/

I had success detecting these patterns and replacing them in a normal zsh (version: zsh 5.7.1 (x86_64-apple-darwin19.0)) terminal window with this line of code

sed -E $'s/infra\\/helm\\/\\*\\/charts\\/(\\*)\*/infra\\/helm\\/\\*\\/charts\\/\\*\\\ninfra\\/helm\\/\\*\\/manifests\\//' myfile

I encountered a strange change in behavior when introducing the '-i' flag to (GNU) sed. When I ran the exact same line, using this flag

(i.e.

sed -iE $'s/infra\\/helm\\/\\*\\/charts\\/(\\*)\*/infra\\/helm\\/\\*\\/charts\\/\\*\\\ninfra\\/helm\\/\\*\\/manifests\\//' datfile

)

I found that the file did not get modified. Normally I would assume this was due to some peculiarity with file permissions or the process being generated by invoking a script instead of a direct terminal entry, but strangely, I've confirmed

sed -iE $'s/infra\\/helm\\/\\*\\/charts\\/\\*/infra\\/helm\\/\\*\\/charts\\/\\*\\\ninfra\\/helm\\/\\*\\/manifests\\//' datfile

Works just fine. (To save eyestrain, the only difference in this line is that it accepts

infra/helm/*/charts/*

as a pattern match, without also accepting

infra/helm/*/charts/

)

Is anyone familiar with why this might be? FWIW, I've confirmed this behavior persists whether I use the aforementioned zsh version as the interpereter, or GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19) Copyright (C) 2007 Free Software Foundation, Inc. as an alternative. It shouldn't be too difficult to find some workaround, but I'd like to improve my sed skills, so learning why this is happening would be really valuable to me.

I've attached a dump of the information from running

strings $(which sed) 

below

$FreeBSD: src/usr.bin/sed/compile.c,v 1.28 2005/08/04 10:05:11 dds Exp $
$FreeBSD: src/usr.bin/sed/main.c,v 1.36 2005/05/10 13:40:50 glebius Exp $
$FreeBSD: src/usr.bin/sed/misc.c,v 1.10 2004/08/09 15:29:41 dds Exp $
$FreeBSD: src/usr.bin/sed/process.c,v 1.39 2005/04/09 14:31:41 stefanf Exp $

Answer 1

GNU sed's -i option takes an optional argument, which is the suffix to use for backups. So sed -iE … datfile causes the old version of the file to be saved as datfileE. It doesn't match what you expect since you aren't passing the -E option, so the regex is parsed as a BRE and not an ERE.

The fix is to pass the options separately: sed -i -E … or sed -E -i …. (-Ei would also work, if you hate the next person who'll read your code.)

Alternatively, pass a backup suffix (sed -E -i.bak …), which lets your script work both with FreeBSD sed and with GNU sed. The suffix is necessary for BSD sed because its -i option has a mandatory argument, and must be in the same command line argument as the option (no space between -i and .bak) so that GNU sed treats it as an optional argument to -i and not a separate argument to sed. This has the side effect of creating backup files.