Python Popen not behaving like a subprocess

Question

My problem is this--I need to get output from a subprocess and I am using the following code to call it-- (Feel free to ignore the long arguments. The importing thing is the stdout= subprocess.PIPE)

        (stdout, stderr) = subprocess.Popen([self.ChapterToolPath, "-x",  book.xmlPath , "-a", book.aacPath , "-o",   book.outputPath+ "/" + fileName + ".m4b"], stdout= subprocess.PIPE).communicate()
        print stdout

Thanks to an answer below, I've been able to get the output of the program, but I still end up waiting for the process to terminate before I get anything. The interesting thing is that in my debugger, there is all sorts of text flying by in the console and it is all ignored. But the moment that anything is written to the console in black (I am using pycharm) the program continues without a problem. Could the main program be waiting for some kind of output in order to move on? This would make sense because I am trying to communicate with it.... Is there a difference between text that I can see in the console and actual text that makes it to the stdout? And how would I collect the text written to the console?

Thanks!

Answer 1

The first line of the documentation for subprocess.call() describes it as such:

Run the command described by args. Wait for command to complete, then return the returncode attribute.

Thus, it necessarily waits for the subprocess to exit.

subprocess.Popen(), by contrast, does not do this, returning a handle on a process with which one than then communicate().

Answer 2

To get all output from a program:

from subprocess import check_output as qx

output = qx([program, arg1, arg2, ...])

To get output while the program is running:

from subprocess import Popen, PIPE

p = Popen([program, arg1, ...], stdout=PIPE)
for line in iter(p.stdout.readline, ''):
    print line,

There might be a buffering issue on the program' side if it prints line-by-line when run interactively but buffers its output if run as a subprocess. There are various solutions depending on your OS or the program e.g., you could run it using pexpect module.