pure shell complex substitution in variable

Question

Is there any way to substitute text in variables on several patterns at time or even using back reference?

For example, I have FILE=filename.ext and I want to change it to filename_sometext.ext. But I don't know that file extension is .ext. All I know about it is that extension is after last dot. So I can do it in two steps:

EXT=${FILE##*.}
FILE=${FILE%.*}_sometext.$EXT

Can I do it on one step (something like ${FILE/.\(*\)/_sometext.\1} [that doesn't work])?

By the way I need to do it in pure shell without sed/awk/etc. My shell is ksh, but if there is way to do it with bashisms I'd like to know it too.

Answer 1

Bash Parameter Expansion says that the variable (FILE in your example) must be a parameter name. So they don't nest. And the last part of ${param/pattern/replacement} must be a string. So back references aren't supported.

My only advice is to use

${EXT:+.$EXT}

to avoid adding a trailing dot if the file has no extension.

---

UPDATE

Apparently back references are supported in ksh93.

So you could use something like

FILE="${FILE/@(.*)/_something\1}"

Answer 2

In this particular case I think FILE=${FILE%.*}_sometext.${FILE##*.} would do the job.

Answer 3

Variables.

In any shell, you could concatenate variables or values:

$ a=One; b=Two; c=Three
$ d="$a$b$c"; echo $d
OneTwoThree

The ${parameter%word} type of structures is POSIX have been supported in most shells for a long time.
Assuming FILE=filename.ext, you could do:

$ FILE="${FILE%.*}_sometext.${FILE##*.}"; echo "$FILE"
filename_sometext.ext

That is one line (as requested) and works on most shells.

---

It is also possible to use variables (even with several dots):

#!/bin/bash
FILE=file.one.name.ext
ADDTEXT="_sometext"
EXT="${FILE##*.}"
echo "EXT=$EXT"
echo "final FILE=${FILE%.*}${ADDTEXT}.$EXT"

Or, in just one line:

ONEFILE=${FILE%.*}${ADDTEXT}.${FILE##*.}
echo "one   FILE=$ONEFILE"

---

Pattern substitution

What gets tricky is trying to use ${parameter/pattern/string} «Pattern substitution.»

A variable works in many shells (even busybox ash):

NEWTEXT="${ADDTEXT}.${FILE##*.}"
echo "ash   FILE=${FILE/.*/${NEWTEXT}}"

But not in older shells.

---

Only for FILE=filename.ext:
This works in bash (and ksh, ksh93, mksh, zsh) but not in ash, dash, sh or csh

echo "bash  FILE=${FILE/%.*/${ADDTEXT}.${FILE##*.}}"

Note that it is using the % character to indicate a match at the end of the parameter (but, sadly it is greedy and eats all dots in FILE=file.one.name.ext).

Conclusion

The best method to control how greedy is the substitution is to use separate variables (as opposed to some from of «Pattern substitution.»)

Answer 4

FILE="${FILE%.*}_sometext${FILE#"${FILE%.*}"}"

If the shell variable $FILE contains a . the above command will reassign to it the old value less everything following the last . then _sometext and last the old value's last . and everything after.

And so:

FILE=some.dot
FILE="${FILE%.*}_sometext${FILE#"${FILE%.*}"}"
printf %s\\n "$FILE"

---

some_sometext.dot

If the shell variable $FILE does not contain a dot, then the _sometext string is merely appended to the end of the old value:

FILE=no_dots_at_all
FILE="${FILE%.*}_sometext${FILE#"${FILE%.*}"}"
printf %s\\n "$FILE"

---

no_dots_at_all_sometext

This is importantly different behavior than you might get if you did not nest the parameter expansion as I do on the end there:

FILE=no_dots_at_all
FILE="${FILE%.*}_sometext${FILE##*.}"
printf %s\\n "$FILE"

---

no_dots_at_all_sometextno_dots_at_all

When you nest the parameter expansion it is evaluated inside to outside, and so the first thing that happens is:

for    FILE in    some.dot no_dots_at_all
do     printf %s\\n '${FILE#'"${FILE%.*}"'}'
done

---

${FILE#some}
${FILE#no_dots_at_all}

...the shell removes any matched bits and uses the remains as the pattern to strip from the outer expansion. And so when there aren't any matched bits and the variable expands in full the outer expansion strips away all of the variable rather than substituting it into your expression twice.