17

I am trying to write a script which is POSIX compliant, so it can run on any *nix system (Debian, Fedora, CentOS, AIX, Arch... ALL of them). When it comes to redirection, I am rather confused on what is supported by POSIX and what is not.

If I redirect ls -l file missingfile &> out.txt this works perfectly in bash. The line from stderr complaining that missingfile doesn't exist, and the permissions output from file, are both in out.txt. However, I believe this is only working because these shells support more than just POSIX standard operators. Upon researching this question I've come across conflicting answers.

This stackexchange answer, for example, seems to imply that &>, >&, >>& and &>> are non-standard operations. However, this answer explicitly states that some-program > some_file 2>&1 is POSIX compliant. Does this mean that the >& operator (using digits for stderr & stdout) is POSIX compliant whereas using &> to auto redirect both stderr and stdout to a file is not POSIX compliant? Or are both &> and >& not POSIX compliant / POSIX compliant (one of these two people are wrong)?

I considered just avoiding the & sign completely and using ls -l file missingfile >out.txt 2>out.txt however this comes with its own issue. Running the ls command this way causes the shell to open two file handles for out.txt, both pointing to offset 0 in the file. So when ls looks for these two files, one of the messages gets clobbered.

What one would expect ls -l file missingfile >out.txt 2>out.txt to output:

-rw-r--r-- 1 blah blah 0 Jun  3 13:18 file
ls: cannot access 'missingfile': No such file or directory

What is actually output:

-rw-r--r-- 1 blah blah 0 Jun  3 13:18 file
le or directory

What is the best way to redirect both stdout and stderr to a file in a POSIX compliant fashion?

Improve this question
5
  • I updated the misleading part of unix.stackexchange.com/a/159514/229446 Commented Jun 3, 2020 at 19:07
  • 2
    &> out.txt doesn't work in Dash. It does in Busybox sh, though, along with Bash and Zsh. Commented Jun 3, 2020 at 19:44
  • @ilkkachu You are correct. I got confused on which shell I was using when I was testing it. I've updated my question Commented Jun 3, 2020 at 22:03
  • 4
    This isn't really about portability across operating systems, but portability across shells. One can get variance in what provides the sh command even simply among versions of the same operating system in some cases. Commented Jun 4, 2020 at 0:57
  • 1
    There is a shell called dash which is meant not to have any extension to POSIX standard. So if your script works in dash, it will work in bash, ksh, ash etc. Commented Jun 7, 2020 at 5:28

1 Answer 1

Reset to default
43

You can check the POSIX specification. The language isn't always easy to follow, but if you know what you're looking for, you can see whether it's there or not.

POSIX includes the redirection operations N>&word and N<&word where N is an optional file descriptor number (defaulting to 0 for input and 1 for output) and word can be:

  • A sequence of digits, to redirect file descriptor N to whatever is already open on word.
  • The character -, to close file descriptor N.

So >out.txt 2>&1 is a POSIX-compliant way to redirect both standard output and standard error to out.txt. It also works in pre-POSIX Bourne shells.

>& followed by a file name is an extension in some shells (including bash) that redirects both stdout and stderr to that file.

&> is a synonym of >& followed by a file name in bash (and zsh). It has the advantage of working even if the file name is a sequence of digits or - (for example foo >&"$a" is equivalent to foo 1>&3 2>&1 if the value of a is the digit 3, but to foo >myfile 2>&1 if the value of a is myfile). Its downside is that it is incompatible with POSIX. In a POSIX-compliant shell, echo foo &>bar is parsed as the command echo foo, the & operator, and the command >bar: a background process that prints foo, and a foreground process that creates or empties the file bar.

Improve this answer
7
  • 1
    Interestingly enough, the POSIX-compliant way (>file.txt 2>&1) also works for Windows' command line. However, instead of - to close the file descriptor, you redirect to nul (same as /dev/null). Commented Jun 4, 2020 at 10:23
  • @IsmaelMiguel Watch out. Does redirection to NUL actually close the fd (thus return errors on I/O, like 2>-), or just discard writes/return EOF on reads (like 2>/dev/null)? Commented Jun 4, 2020 at 14:01
  • 1
    @TooTea As I said, you use nul to redirect the output stream, not to close it. nul is the same as /dev/null. From what I've seen, there's no file descriptor closing. ss64.com/nt/syntax-redirection.html <-- should give you a better light on what I mean Commented Jun 4, 2020 at 14:14
  • In the last paragraph, shouldn't it read &> filename? Commented Jun 4, 2020 at 19:36
  • 1
    @Xerus Sometimes. It isn't the same thing, but in some circumstances it has the same effect. >/dev/null redirects the file descriptor whereas >- closes it. If the application tries to write and ignores write failures, this has the same effect. If the application exits on write failures, this likely doesn't have the same effect. If the application opens a file and it lands on that descriptor, this could have a completely different effect, e.g. ending up writing log messages to the file intended for normal output. Commented May 7, 2021 at 10:18

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.