POSIX shells
The usual (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ) trick to get the complete stdout of a command is to do:
output=$(cmd; ret=$?; echo .; exit "$ret")
ret=$?
output=${output%.}
The idea is to add an extra .\n. Command substitution will only strip that \n. And you strip the . with ${output%.}.
Note that in shells other than zsh, that will still not work if the output has NUL bytes. With yash, that won't work if the output is not text.
Also note that in some locales, it matters what character you use to insert at the end. . should generally be fine (see below), but some other might not. For instance x (as used in some other answers) or @ would not work in a locale using the BIG5, GB18030 or BIG5HKSCS charsets. In those charsets, the encoding of a number of characters ends in the same byte as the encoding of x or @ (0x78, 0x40)
For instance, ū in BIG5HKSCS is 0x88 0x78 (and x is 0x78 like in ASCII, all charsets on a system must have the same encoding for all the characters of the portable character set which includes English letters, @ and .). So if cmd was printf '\x88' (which by itself is not a valid character in that encoding, but just a byte-sequence) and we inserted x after it, ${output%x} would fail to strip that x as $output would actually contain ū (the two bytes making up a byte sequence that is a valid character in that encoding).
Using . or / should be generally fine, as POSIX requires:
- “The encoded values associated with
<period>,<slash>,<newline>, and<carriage-return>shall be invariant across all locales supported by the implementation.”, which means that these will have the same binary represenation in any locale/encoding. - “Likewise, the byte values used to encode
<period>,<slash>,<newline>, and<carriage-return>shall not occur as part of any other character in any locale.”, which means that the above cannot happen, as no partial byte sequence could be completed by these bytes/characters to a valid character in any locale/encoding. (see 6.1 Portable Character Set)
The above does not apply to other characters of the Portable Character Set.
Another approach, as discussed by @Isaac, would be to change the locale to C (which would also guarantee that any single byte can be correctly stripped), only for the stripping of the last character (${output%.}).
It would be typically necessary to use LC_ALL for that (in principle LC_CTYPE would be enough, but that could be accidentally overridden by any already set LC_ALL). Also it would be necessary to restore the original value (or e.g. the non-POSIX compliant locale be used in a function). But beware, that some shells don't support changing the locale while running (though this is required by POSIX).
By using . or /, all that can be avoided.
bash/zsh alternatives
With bash and zsh, assuming the output has no NULs, you can also do:
IFS= read -rd '' output < <(cmd)
To get the exit status of cmd, you can do wait "$!"; ret=$? in bash but not in zsh.
rc/es/akanaga
For completeness, note that rc/es/akanga have an operator for that. In them, command substitution, expressed as `cmd (or `{cmd} for more complex commands) returns a list (by splitting on $ifs, space-tab-newline by default). In those shells (as opposed to Bourne-like shells), the stripping of newline is only done as part of that $ifs splitting. So you can either empty $ifs or use the ``(seps){cmd} form where you specify the separators:
ifs = ''; output = `cmd
or:
output = ``()cmd
In any case, the exit status of the command is lost. You'd need to embed it in the output and extract it afterwards which would become ugly.
fish
In fish, command substitution is with (cmd) and doesn't involve a subshell.
set var (cmd)
Creates a $var array with all the lines in the output of cmd if $IFS is non-empty, or with the output of cmd stripped of up to one (as opposed to all in most other shells) newline character if $IFS is empty.
So there's still an issue in that (printf 'a\nb') and (printf 'a\nb\n') expand to the same thing even with an empty $IFS.
To work around that, the best I could come up with was:
function exact_output
set -l IFS . # non-empty IFS
set -l ret
set -l lines (
cmd
set ret $status
echo
)
set -g output ''
set -l line
test (count $lines) -le 1; or for line in $lines[1..-2]
set output $output$line\n
end
set output $output$lines[-1]
return $ret
end
An alternative is to do:
read -z output < (begin; cmd; set ret $status; end | psub)
Since version 3.4.0 (released in March 2022), fish also supports $(...) which behaves like (...) except that it can also be used inside double quotes in which case it behaves like in the POSIX shell: the output is not split on lines but all trailing newline characters are removed.
Bourne shell
The Bourne shell did not support the $(...) form nor the ${var%pattern} operator, so it can be quite hard to achieve there. One approach is to use eval and quoting:
eval "
output='`
exec 4>&1
ret=\`
exec 3>&1 >&4 4>&-
(cmd 3>&-; echo \"\$?\" >&3; printf \"'\") |
awk 3>&- -v RS=\\\\' -v ORS= -v b='\\\\\\\\' '
NR > 1 {print RS b RS RS}; {print}; END {print RS}'
\`
echo \";ret=\$ret\"
`"
Here, we're generating a
output='output of cmd
with the single quotes escaped as '\''
';ret=X
to be passed to eval. As for the POSIX approach, if ' was one of those characters whose encoding can be found at the end of other characters, we'd have a problem (a much worse one as it would become a command injection vulnerability), but thankfully, like ., it's not one of those, and that quoting technique is generally the one that is used by anything that quotes shell code (note that \ has the issue, so shouldn't be used (also excludes "..." inside which you need to use backslashes for some characters). Here, we're only using it after a ' which is OK).
tcsh
See tcsh preserve newlines in command substitution `...`
(not taking care of the exit status, which you could address by saving it in a temporary file (echo $status > $tempfile:q after the command))