How to capture output from continuous process in Python?

Question

I am new to Python and Linux. I have a process running in a terminal window that will go indefinitely. The only way to stop it would be for it to crash or for me to hit ctrl+C. This process outputs text to the terminal window that I wish to capture with Python, so I can do some additional processing of that text.

I know I need to do something with getting stdout, but no matter what I try, I can't seem to capture the stdout correctly. Here is what I have so far.

import subprocess

command = 'echo this is a test. Does it come out as a single line?'

def myrun(cmd):
    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

    stdout = []
    while True:
        line = p.stdout.read()
        stdout.append(line)

        if line == '' and p.poll() != None:
                break
    return ''.join(stdout)

result = myrun(command)
print('> ' + result),

This will work when my command is a simple "echo blah blah blah". I am guessing this is because the echo process terminates. If I try running the continuous command, the output is never captured. Is this possible to do?

Perhaps you are struggling with the fact Python buffers it's output, try running your script with python -u script.py and see if it works — Havenard
– Havenard, Commented Aug 22, 2018 at 3:30

georgexsh · Accepted Answer · 2018-08-22 05:41:55Z

2

read() will block on reading until reach EOF, use read(1024) or readline() instead:

read(size=-1)

Read and return up to size bytes. If the argument is omitted, None, or negative, data is read and returned until EOF is reached.

eg:

p = subprocess.Popen('yes', stdout=subprocess.PIPE)

while True:
    line = p.stdout.readline()
    print(line.strip())

see more on the python io doc.

edited Aug 22, 2018 at 5:41

answered Aug 22, 2018 at 5:28

georgexsh

16.8k3 gold badges45 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to capture output from continuous process in Python?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related