how to tell, from the outside, if the output stream of a process has been closed?

Question

Suppose you have this script, on an Ubuntu machine:

#!/bin/env python3
import sys
import time

try:
  i = 1
  while True:
    print(i)
    i += 1
except Exception as e:
  sys.stderr.write(f"We caught an exception {e!r}\n")
  sys.stderr.flush()

while True:
  sys.stderr.write("Sleeping for a minute\n")
  sys.stderr.flush()
  time.sleep(60)

And you run it like this:

# ./some_test.py | head -n 10
1
2
3
4
5
6
7
8
9
10
We caught an exception BrokenPipeError(32, 'Broken pipe')
Sleeping for a minute
Sleeping for a minute
.
.
.

And it will not stop, of course. Is there a way in which, from the outside, we can see that the output stream is closed? I checked with lsof and and also fds in /proc and I do not see anything that makes me think that the stream is closed even though I know that the pipe is broken and the exception when it tried to write also says so:

# lsof -p 28466
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
python3 28466 root  cwd    DIR  252,3     4096      132 /root
python3 28466 root  rtd    DIR  252,3     4096      128 /
python3 28466 root  txt    REG  252,3  5904904     9540 /usr/bin/python3.10
python3 28466 root  mem    REG  252,3  5712144 33793209 /usr/lib/locale/locale-archive
python3 28466 root  mem    REG  252,3  2220400 50352186 /usr/lib/x86_64-linux-gnu/libc.so.6
python3 28466 root  mem    REG  252,3   108936 50422136 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
python3 28466 root  mem    REG  252,3   194872 50417841 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7
python3 28466 root  mem    REG  252,3   940560 50413373 /usr/lib/x86_64-linux-gnu/libm.so.6
python3 28466 root  mem    REG  252,3    27002 50386759 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
python3 28466 root  mem    REG  252,3   240936 50359529 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
python3 28466 root    0u   CHR  136,3      0t0        6 /dev/pts/3
python3 28466 root    1w  FIFO   0,13      0t0   110475 pipe
python3 28466 root    2u   CHR  136,3      0t0        6 /dev/pts/3

# ls -l /proc/28466/fd
total 0
lrwx------ 1 root root 64 Oct 20 13:44 0 -> /dev/pts/3
l-wx------ 1 root root 64 Oct 20 13:44 1 -> 'pipe:[110475]'
lrwx------ 1 root root 64 Oct 20 13:44 2 -> /dev/pts/3

Answer 1

The stream isn’t closed; the fact that you are able to try writing to it without receiving an EBADF error is proof of that. (See man 2 write for details.)

On Linux, given that you’re looking at a pipe, what you could do is check all running processes to see if any other has an open file descriptor to the same pipe. If you find one, then the pipe is still usable for writing without error; if you don’t then either you don’t have appropriate privileges to see the reading process, or the pipe is no longer usable for writing.

lsof can be used for that. Example:

$ sleep 3m | sleep 10 &
[1] 62807 62808
$ lsof -p 62807 -ad 1 +E
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
sleep   62807 stephane    1w  FIFO   0,14      0t0 806533 pipe 62808,sleep,0r
sleep   62808 stephane    0r  FIFO   0,14      0t0 806533 pipe 62807,sleep,1w

Stdout of 62807 (running sleep 3m) is open on the writing end of a pipe, and there's still a process (running sleep 10) with a fd (here 0) open on the reading end.

Same command run after 10 seconds:

$ lsof -p 62807 -ad 1 +E
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
sleep   62807 stephane    1w  FIFO   0,14      0t0 806533 pipe

This time, lsof can't find any other fd open for reading on the pipe upon on fd 1 of 62807.

Without lsof, you can do the equivalent of this zsh code:

$ sleep 3m | sleep 4m &
[1] 63098 63100
$ ls -ogd /proc/*/fd/*(e['[[ $REPLY -ef /proc/63098/fd/1 ]]'])
l-wx------ 1 64 Oct 20 17:12 /proc/63098/fd/1 -> 'pipe:[810032]'
lr-x------ 1 64 Oct 20 17:12 /proc/63100/fd/0 -> 'pipe:[810032]'

Permissions of those symlinks indicate how the pipe is open for each process.

See also How to find out if pipe is broken?

Answer 2

Update: After re-reading your question, I see that you were asking about monitoring it externally. Using lsof or looking in /proc/$PID/fd/$FD for the pipe info and trying to find a matching pair elsewhere under /proc is the only way to really do that. If you don't find a reader and writer matching pair, then either one-half of the pipe has been closed or, in the special case of a named pipe, it could be that its peer has not yet opened the file. In general though, you don't normally run into this hanging a process unless the EPIPE error is being ignored for some reason such as in the original example code.

You've already received your answer. The Python exception BrokenPipeError tells you that the reader on the other end of the pipe has closed their end of the pipe. This corresponds to your Python code ultimately calling the write() system call and receiving an error with the value EPIPE. Once you receive this exception, you should assume that no more output will be accepted and finish the job. Your file descriptor will stay open until you decide to close it, but the other end has already closed their file descriptor.