What's the difference between ">&1" and ">/proc/self/fd/1" redirection?

Question

I'm working on some script that being run by rc.local at startup, and I noticed that output redirection works quite strange.

If I write something like echo "foo" >&1, it ends up in syslog, and all is okay.

But when I write echo "foo" >>/dev/stdout, or echo "foo" >>/proc/self/fd/1, I got an error saying that there is no such device or address.

Further investigation revealed that /proc/self/fd/1 was in fact a socket. Including ls -l /proc/self/fd in rc.local prints the following:

lr-x------. 1 root root 64 Jul 14 05:28 0 -> /dev/null
lrwx------. 1 root root 64 Jul 14 05:28 1 -> socket:[18485]
lrwx------. 1 root root 64 Jul 14 05:28 2 -> socket:[18485]
lr-x------. 1 root root 64 Jul 14 05:28 255 -> /etc/rc.d/rc.local

---

The described behaviour can be easily reproduced with nc. First, find out which file descriptor binds to socket. You can do it by issuing nc -l -p 25566 -c "ls -l /proc/self/fd" in first terminal and then telnet localhost 25566 in other terminal. In my case it was 5-th descriptor.

Okay. Then, to reproduce the issue, in first terminal:

nc -l -p 25566 -c "echo 'hello' >&5"

in second terminal:

telnet localhost 25566

You shall see "hello" in second terminal output between telnet messages about connection being established and closed.

Now the case with file descriptor pseudo-file from /proc:

first terminal:

nc -l -p 25566 -c "echo 'hello' >/proc/self/fd/5"

second terminal:

telnet localhost 25566

Now second terminal just contains telnet messages about connection established and immediately closed, and first terminal shows an error: sh: /proc/self/fd/5: No such device or address.

Edit: OS is Fedora 23 i686 server

---

So, the question. What's the difference between >&1 and >/proc/self/fd/1 ? And is there some universal and reliable way to redirect output to some file, which corresponds exactly to current standard output?

Thank you.

---

Edit 2:

For clarity, exact input/output for Fedora 23 i686 for above case:

Terminal 1                                       | Terminal 2
$ nc -l -p 25566 -c 'ls -go /proc/self/fd'       | 
                                                 | $ telnet localhost 25566
                                                 | Trying ::1...
                                                 | Connected to localhost.
                                                 | Escape character is '^]'.
                                                 | total 0
                                                 | lr-x------ 1 64 Jul 14 08:56 0 -> pipe:[19687]
                                                 | l-wx------ 1 64 Jul 14 08:56 1 -> pipe:[19688]
                                                 | lrwx------ 1 64 Jul 14 08:56 2 -> /dev/tty2
                                                 | lr-x------ 1 64 Jul 14 08:56 3 -> pipe:[19687]
                                                 | lr-x------ 1 64 Jul 14 08:56 4 -> /proc/1285/fd
                                                 | lrwx------ 1 64 Jul 14 08:56 5 -> socket:[19686]
                                                 | l-wx------ 1 64 Jul 14 08:56 7 -> /pipe:[19688]
                                                 | Connection closed by foreign host.
$ nc -l -p 25566 -c "echo 'hi' >&5"              | 
                                                 | $ telnet localhost 25566
                                                 | Trying ::1...
                                                 | Connected to localhost.
                                                 | Escape character is '^]'.
                                                 | hi
                                                 | Connection closed by foreign host.
$ nc -l -p 25566 -c "echo 'hi' >/proc/self/fd/5" | 
                                                 | $ telnet localhost 25566
                                                 | Trying ::1...
                                                 | Connected to localhost.
sh: /proc/self/fd/5: No such device or address   | Escape character is '^]'.
                                                 | Connection closed by foreign host.
$                                                | $

Answer 1

Use >&N. It's portable and as you saw, actually works with sockets.

The only reason you would ever use /proc/self/fd is you are running a program that expects a file name and can't be told to use an already open file descriptor. E.g. the <(cmd...) redirection uses that, since almost all command line utilities can open a file pointed to by name, but not all support preopened file descriptors directly.

Your shell can use pre-existing file descriptors, though, so no need to go through /proc.

---

Also, /proc/NNN/fd/ is specific to Linux, and requires that you have a mounted /proc. On my Linux boxes, /dev/stdout, /dev/fd/* and others are symlinks to /proc/self/fd/* etc., so they require /proc too. On other Unixes, they might be different. According to the answers to an older question /dev/stdout are specifically listed as outside POSIX.

As for why the redirection doesn't work as you tried: Trying out with strace, the difference between the two is that for a >&N redirection, bash calls dup() on the file descriptor, and for >/proc/self/fd/N it just tries to open it as an ordinary file with open(). Apparently proc doesn't support opening new copies of sockets like that, so the call fails. Stream sockets are pretty much point to point links, so prohibiting opening of a new copy doesn't seem too unnatural. But why it works for pipes or with a dup, I couldn't tell.

$ ls -l /proc/self/fd/3
lrwx------ 1 itvirta itvirta 64 Jul 14 18:24 /proc/self/fd/3 -> socket:[168157]
$ strace bash -c "echo foo >>/proc/self/fd/3" 2>&1 | grep open.*proc/self
open("/proc/self/fd/3", O_WRONLY|O_CREAT|O_APPEND, 0666) = -1 ENXIO (No such device or address)

Also this answer has some information about the portability of /proc/NNN/fd

Answer 2

A redirection like 3>&1 duplicates an existing file descriptor: this takes the same open file (same file, same flags, same position, etc.) and plugs it onto another “output port” of the program (another file descriptor number). (More precisely, this creates a new file descriptor that points to the same file description, but we don't)

>&1 duplicates the descriptor onto itself, some shells optimize it away altogether.

A redirection like >/proc/$pid/fd/1 opens the file /proc/$pid/fd/1. This creates a fresh file descriptor. The files in /proc/*/fd are special, and opening them mostly copies the data from an existing file descriptor. Although the files are symbolic links, they are “magic”; for example, a deleted file or a pipe appears as a broken symlink but can still be opened as if it was an existing file because the kernel contains special code to handle /proc/*/fd entries. So in most cases, >&1 and >/proc/self/fd/1 are equivalent. However, sockets get different treatment.

/*
 *      In theory you can't get an open on this inode, but /proc provides
 *      a back door. Remember to keep it shut otherwise you'll let the
 *      creepy crawlies in.
 */

static int sock_no_open(struct inode *irrelevant, struct file *dontcare)
{
        return -ENXIO;
}

(The code has been reorganized in recent versions, but sockets still can't be opened even via /proc/PID/fd/NUM.)

The reason you can't open a socket like most other file types is that there's more information attached to a socket, you have to say how you open it. For example, opening a TCP socket allocates a source port. Although it might make sense in some contexts, the Linux kernel doesn't allow opening another process's sockets. In the case where the process is the same (/proc/self/fd/NUM, as opposed to /proc/OTHERPID/fd/NUM), it could convert the open call to the usual file descriptor duplication, but opening /proc/self/fd is an unusual thing to do in the first place, and redirection with sockets isn't normally done since it normally doesn't work, so the kernel has not been designed to support this sensible-but-useless exception.