Bash process substitution: how to portably refer to standard input?

Question

The question here is the bold sentence below. Suppose I'm trying to do this:

diff -u <(some command) {mystery-syntax}

I am diffing the output of some command and some data that comes from standard input—for instance, a copy-and-paste in the terminal. Like this:

$ diff -u <(echo foo) {mystery-syntax}
bar
[Ctrl-D]
--- /dev/fd/63  2021-11-04 11:28:19.360366909 -0700
+++ /dev/fd/62  2021-11-04 11:28:19.360366909 -0700
@@ -1 +1 @@
-foo
+bar

What actual syntax is the {mystery-syntax} meta-token covering?

The diff utility doesn't use the - convention for specifying standard input, so we cannot use that.

One answer is: /dev/fd/0.

$ diff -u <(echo foo) /dev/fd/0
bar
[Ctrl-D]
--- /dev/fd/63  2021-11-04 11:31:02.837040697 -0700
+++ /dev/fd/0   2021-11-04 11:30:56.807964644 -0700
@@ -1 +1 @@
-foo
+bar

OK, so we have it working in one way, on systems that have Linux-style /dev/fd. But is there a way without referencing the /dev filesystem, which is a system-specific implementation detail? I believe Bash process substitution is supported on systems which have mechanisms other than /dev/fd. The manual states that:

Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files.

For instance, the obvious:

diff <(echo foo) <(cat)

doesn't work; for whatever reason, even though the <(cat) process substitution needs only the output side of cat, cat is nevertheless being invoked in such a way that it has no standard input:

$ diff -u <(echo foo) <(cat)
cat: -: Input/output error
--- /dev/fd/63  2021-11-04 11:33:59.303347814 -0700
+++ /dev/fd/62  2021-11-04 11:33:59.303347814 -0700  
@@ -1 +0,0 @@
-foo

There are some "clever" behaviors here, too. If we explicitly redirect /dev/fd/0 into cat's input, that is still borked somehow:

$ diff -u <(echo foo) <(cat < /dev/fd/0)
cat: -: Input/output error

But! If we give cat it a here-document as its standard input, there is no interference:

$ diff -u <(echo foo) <(cat <<<"bar")
--- /dev/fd/63    2021-11-04 11:40:03.612616179 -0700
+++ /dev/fd/62    2021-11-04 11:40:03.612616179 -0700
@@ -1 +1 @@
-foo
+bar

Answer 1

If one of the filenames given to diff on the command line is - (a dash), the utility will read from standard input in its place.

$ echo hello | diff -u <( echo ok ) -
--- /dev/fd/12  Fri Nov 12 21:25:18 2021
+++ -   Fri Nov 12 21:25:18 2021
@@ -1 +1 @@
-ok
+hello

From the GNU diff manual:

If a FILE is -, read standard input.

From the OpenBSD manual:

If either file1 or file2 is -, the standard input is used in its place.

From the POSIX standard:

file1, file2
A pathname of a file to be compared. If either the file1 or file2 operand is -, the standard input shall be used in its place.

Answer 2

diff -u <(echo foo) <(cat) doesn't work in an interactive bash because commands in process substitution are not put in the foreground so can't read from the terminal device. Like in cat & where cat is also run in background (not in the foreground process group of the terminal device), the process running cat is suspended with a SIGTTIN signal as soon as it tries to read from its controlling terminal.

echo hello | diff -u <(echo ok) <(cat) works because the whole pipeline made of two subshells is in foreground, and anyway cat is reading from the pipe, not the terminal. The process substitutions end up being in foreground because they were started by the subshell (the second component of that pipeline) that is in foreground.

bash-5.0$ diff -u <(echo ok) <(ps -o pid,ppid,pgid,args)
--- /dev/fd/63  2021-11-12 22:39:41.985207894 +0000
+++ /dev/fd/62  2021-11-12 22:39:41.985207894 +0000
@@ -1 +1,6 @@
-ok
+    PID    PPID    PGID COMMAND
+   9779    9772    9779 /bin/zsh
+1957388    9779 1957388 bash --norc
+1958861 1957388 1957388 bash --norc
+1958862 1957388 1958862 diff -u /dev/fd/63 /dev/fd/62
+1958863 1958861 1957388 ps -o pid,ppid,pgid,args

--- /dev/fd/63  2021-11-12 22:39:01.017237420 +0000
+++ /dev/fd/62  2021-11-12 22:39:01.017237420 +0000
@@ -1 +1,7 @@
-ok
+    PID    PPID    PGID COMMAND
+   9779    9772    9779 /bin/zsh
+1957388    9779 1957388 bash --norc
+1958325 1957388 1958324 diff -u /dev/fd/63 /dev/fd/62
+1958326 1958325 1958324 [bash] <defunct>
+1958327 1958325 1958324 bash --norc
+1958328 1958327 1958324 ps -o pid,ppid,pgid,args

See how ps is running in a different process group in the first case above, and in the same process group as the rest in the second case.

If you run:

bash-5.0$ (diff -u <(echo ok) <(ps -o pid,ppid,pgid,args))
--- /dev/fd/63  2021-11-12 22:42:04.761112701 +0000
+++ /dev/fd/62  2021-11-12 22:42:04.761112701 +0000
@@ -1 +1,6 @@
-ok
+    PID    PPID    PGID COMMAND
+1960605    9772 1960605 /bin/zsh
+1960612 1960605 1960612 bash --norc
+1960757 1960612 1960757 diff -u /dev/fd/63 /dev/fd/62
+1960759 1960757 1960757 bash --norc
+1960760 1960759 1960757 ps -o pid,ppid,pgid,args

It's back to being OK as then we start a whole subshell in foreground.

So the easy work around here is to run the pipeline in a subshell.

In any case, again, that only applies to interactive shells (where terminal job control is being performed) and when stdin is the controlling terminal of the session.

In scripts (and you generally only care about portability in scripts), there's no problem with diff -u <(echo ok) <(cat) even when stdin is a terminal as the shell is non-interactive and doesn't do job control.

That cat however does nothing useful, it's the same UUOC as in:

cat | diff -u <(echo ok) -

Or

cat | diff -u <(echo ok) /dev/stdin

On systems that don't support /dev/fd/x / /dev/stdin¹, for commands which unlike diff, don't support - to specify stdin, <(cat) in bash or zsh (but not in ksh versions prior to ksh93u where that feature comes from) uses a temporary named pipe instead, so diff -u <(echo ok) <(cat) will still work on those systems.

Since we mention portability though, note that in zsh, you need:

echo hello | { diff <(echo ok) <(cat); }

For the stdin of cat to also be the pipe from echo.

Also note that:

echo hello | cmd /dev/stdin # or /dev/fd/0, /proc/self/fd/0²

Doesn't work with ksh93 on Linux, because there, ksh93's | uses socketpairs instead of pipes³ and on Linux (and Cygwin), contrary to other systems, opening /dev/stdin is not like dup(0), it opens the same file as open on stdin, and you can't open() a socket.

More reading in connected Q&As here:

• What is the portable (POSIX) way to achieve process substitution?
• When was process substitution first introduced?
• Bash: process substitution and stdin

---

^{¹ very few these days and there are probably far more systems without bash than without /dev/fd/x}

^{² except for those commands such as gawk that treat or can treat /dev/stdin as meaning stdin by themselves without actually opening the file in the same was as for -.}

^{³ as Linux' pipe implementation doesn't let you peek at the contents which ksh93 needs to perform some (dubious) optimisations.}