Process substitution and redirection using tee

Question

Suppose that you redirect, in bash, the standard output of a command cmd to a file named f.out, and the standard error to f.err, using tee to preserve console printing:

cmd 1> >(tee f.out) 2> >(tee f.err)

Then f.out contains the output as well as the error (at least on my system).

Now, if you change the order of redirections:

cmd 2> >(tee f.err) 1> >(tee f.out)

f.out only contains the output (and f.err only contains the error in both cases).

So my question is double: how stderr can be redirected to f.out, and why does the order of redirections impact the result?

Note that if you don't use tee, but for example cat, like this:

cmd 1> >(cat>f.out) 2> >(cat>f.err)

you don't have this issue, and the order of redirections doesn't matter, as expected, and as it would be the case without process substitution (cmd 1>f.out 2>f.err).

Answer 1

Order of redirection is important because Bash applies them in the order it finds them on the command it interprets.

This is on purpose so that you can have idioms like > file 2>&1 working as expected i.e. having stderr the same as stdout. This idiom works as in "assign file to stdout and then make stderr equal to stdout", which yields the expected outcome because by the time stderr gets stdout's same value, stdout's value is file. The other way around (ie 2>&1 1> file) won't yield the same outcome because stdout's value is changed after it has been copied to stderr's value. File-descriptors can be considered analogous to regular variables, which have their own values and can be made to get a copy of another variable's value, as in var1="${var2}", and much like such var1 won't follow var2's subsequent value changes, file-descriptor's value won't too.

It is also handy so that you can e.g. swap file-descriptors on the same line, like in 3>&1 1>&2 2>&3-. This swaps fds 1 and 2 using fd 3 as temporary “helper” fd.

As such you can consider redirections as instructions executed sequentially, just as if they were on two separate lines of your command or script.

On your specific case there are also Process Substitutions involved, and those too get executed in the specified sequence inheriting the redirections expressed up to that point

That is, to cap it all:

1. you first redirect stdout to the process running tee f.out; at this point cmd’s stdout is connected to tee f.out’s stdin, as desired
2. then you redirect stderr to the process running tee f.err; but this inherits its stdout as per the redirection expressed before, i.e. connected to tee f.out’s stdin

Therefore tee f.err, by innocently outputting to its stdout as well as to f.err file, pipes your cmd’s error messages to tee f.out’s stdin which will therefore receive all messages, outputting them to f.out file as well as to your terminal window.

Answer 2

I came across this question which was part of the answer to a test I am working on. The general case of the command is to run a cmd and display on terminal, with stderr modified (indented & coloured red) and then save what appears on screen in a log file. Removing the colour coding from the log.

So, the example cmd creating 2 stdout and 2 stderr lines, is

{ ls -l /usr/bin/col{1,2,a,b} 2>&1 >&3 | sed 's/.*/\t\o33[31mSTDERR: &\o33[0m/'; } 3>&1  

Now, putting a copy into a log file

{ ls -l /usr/bin/col{1,2,a,b} 2>&1 >&3 | sed 's/.*/\t\o33[31mSTDERR: &\o33[0m/'; } 3>&1 | tee >( sed 's/\o33//g;s/\[31m//g;s/\[0m//' > /tmp/file )

This works fine, however; is there a better way to achieve this?