FIFO capture using cat not working as intended?

Question

Hi I am trying to use a Unix FIFO to communicate between a Python script and a shell script. The intention is for the shell script to capture all output of the python script. So far I have the following:

#!/bin/bash

# Test IPC using FIFOs.

## Create a named pipe (FIFO).
mkfifo ./myfifo

## Launch the Python script asynchronously and re-direct its output to the FIFO.
python3 helloworld.py > ./myfifo &
PID_PY=$!
echo "Python script (PID=$PID_PY) launched."

## Read from the FIFO using cat asynchronously.
## Note that running asynchronously using & runs it the program (in this case `cat`)
## in a child shell "subshell", so I will collect the output in a file.
echo "Reading FIFO."
>output.log cat ./myfifo &
PID_CAT=$!

## Sleep for 10 seconds.
sleep 10

## Kill the Python script.
kill -15 $PID_PY && echo "Python script (PID=$PID_PY) killed."

## Kill the cat!
kill -15 $PID_CAT

## Remove the pipe when done.
rm -fv ./myfifo

## Check for the existence of the output log file and print it.
[[ -f output.log ]] && cat output.log || echo "No logfile found!." 1>&2

However when I open the log file output.log, it is empty which is why the last command returns empty. Is there something I am doing wrong. I understand the above might be easily accomplished using an anonymous pipe like so: python3 helloworld.py | cat >output.log (or even python3 helloworld.py > output.log for that matter) but my intention is to understand the use of named pipes in Unix/Linux.

The python script just prints something to stdout every 1 second:

if __name__ == "__main__":
    import time
    try:
        while True:
            print("Hello, World")
            time.sleep(1)
    except KeyboardInterrupt:
        print('Exiting.')
    finally:
        pass

Answer 1

Your understanding of how fifos work is fine, the problem is with the python script. As with any such thing, the best way to debug is to run the commands in the terminal manually so you can see what each is doing. If you run helloworld.py > outfile, and then cat outfile, you will see it has no contents. If, however, you don't redirect the output and just run helloworld.py, you get normal output. This is because Python buffers its output and hasn't had time to fill the buffer and print out its contents by the time you kill the script.

If you replace your helloworld.py with a simple shell script, it works as expected:

$ cat helloworld.sh
#!/bin/sh

while :; do
  echo "hello world"
  sleep 1
done

Alternatively, make your python produce unbuffered output, and it also works as expected:

python3 -u helloworld.py > ./myfifo &