How do I write to a Python subprocess' stdin?

Since subprocess 3.5, there is the subprocess.run() function, which provides a convenient way to initialize and interact with Popen() objects. run() takes an optional input argument, through which you can pass things to stdin (like you would using Popen.communicate(), but all in one go).

Adapting jro's example to use run() would look like:

import subprocess
p = subprocess.run(['myapp'], input='data_to_write', capture_output=True, text=True)

After execution, p will be a CompletedProcess object. By setting capture_output to True, we make available a p.stdout attribute which gives us access to the output, if we care about it. text=True tells it to work with regular strings rather than bytes. If you want, you might also add the argument check=True to make it throw an error if the exit status (accessible regardless via p.returncode) isn't 0.

This is the "modern"/quick and easy way to do to this.

To clarify some points:

As jro has mentioned, the right way is to use subprocess.communicate.

Yet, when feeding the stdin using subprocess.communicate with input, you need to initiate the subprocess with stdin=subprocess.PIPE according to the docs.

Note that if you want to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE. Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

Also qed has mentioned in the comments that for Python 3.4 you need to encode the string, meaning you need to pass Bytes to the input rather than a string. This is not entirely true. According to the docs, if the streams were opened in text mode, the input should be a string (source is the same page).

If streams were opened in text mode, input must be a string. Otherwise, it must be bytes.

So, if the streams were not opened explicitly in text mode, then something like below should work:

import subprocess
command = ['myapp', '--arg1', 'value_for_arg1']
p = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = p.communicate(input='some data'.encode())[0]

I've left the stderr value above deliberately as STDOUT as an example.

That being said, sometimes you might want the output of another process rather than building it up from scratch. Let's say you want to run the equivalent of echo -n 'CATCH\nme' | grep -i catch | wc -m. This should normally return the number characters in 'CATCH' plus a newline character, which results in 6. The point of the echo here is to feed the CATCH\nme data to grep. So we can feed the data to grep with stdin in the Python subprocess chain as a variable, and then pass the stdout as a PIPE to the wc process' stdin (in the meantime, get rid of the extra newline character):

import subprocess

what_to_catch = 'catch'
what_to_feed = 'CATCH\nme'

# We create the first subprocess, note that we need stdin=PIPE and stdout=PIPE
p1 = subprocess.Popen(['grep', '-i', what_to_catch], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

# We immediately run the first subprocess and get the result
# Note that we encode the data, otherwise we'd get a TypeError
p1_out = p1.communicate(input=what_to_feed.encode())[0]

# Well the result includes an '\n' at the end, 
# if we want to get rid of it in a VERY hacky way
p1_out = p1_out.decode().strip().encode()

# We create the second subprocess, note that we need stdin=PIPE
p2 = subprocess.Popen(['wc', '-m'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)

# We run the second subprocess feeding it with the first subprocess' output.
# We decode the output to convert to a string
# We still have a '\n', so we strip that out
output = p2.communicate(input=p1_out)[0].decode().strip()

This is somewhat different than the response here, where you pipe two processes directly without adding data directly in Python.

Hope that helps someone out.

One can write data to the subprocess object on-the-fly, instead of collecting all the input in a string beforehand to pass through the communicate() method.

This example sends a list of animals names to the Unix utility sort, and sends the output to standard output.

import sys, subprocess
p = subprocess.Popen('sort', stdin=subprocess.PIPE, stdout=sys.stdout)
for v in ('dog','cat','mouse','cow','mule','chicken','bear','robin'):
    p.stdin.write( v.encode() + b'\n' )
p.communicate()

Note that writing to the process is done via p.stdin.write(v.encode()). I tried using print(v.encode(), file=p.stdin), but that failed with the message TypeError: a bytes-like object is required, not 'str'. I haven't figured out how to get print() to work with this.

while .communicate() is the recommended way, it doesnot solve all usecases. For those who wants to write a stream of lines and read its transformation back from a subprocess, here is an example code.

import sys
from pathlib import Path
import itertools
import subprocess
import threading

def copy_to_stdin(proc, src_file: Path, mt_file: Path):
    """Example task: Write data to subproc stdin. 
      Note: run this on another thread to avoid deadlocks
    This function reads two parallel files (src_file and mt_file), and write them as TSV record to the stdin of the sub process.
    :param proc: subprocess object to write to
    :param src_file: path to source file
    :param mt_file: path to MT file
    """

    with src_file.open() as src_lines, mt_file.open() as mt_lines:
        for src_line, mt_line in itertools.zip_longest(src_lines, mt_lines):
            if src_line is None or mt_line is None:
                log.error(f'Input files have different number of lines')
                raise ValueError('Input files have different number of lines')
            line = src_line.rstrip('\n') + '\t' + mt_line.rstrip('\n') + '\n'
            proc.stdin.write(line)
    proc.stdin.flush()
    proc.stdin.close()    # close stdin to signal end of input


cmd_line = ['yourcmd', 'arg1']  # fill your args
src_file, mt_file = ... # your files

proc = subprocess.Popen(cmd_line, shell=False, 
    stdout=subprocess.PIPE, stdin=subprocess.PIPE,
    stderr=sys.stderr, text=True, encoding='utf8', errors='replace') 
try:
    copy_thread = threading.Thread(target=copy_to_stdin, args=(proc, src_file, mt_file))
    copy_thread.start()
    # demonstration of reading data from stdout. 
    for line in proc.stdout:
        line = line.rstrip()
         print(line)
    
    copy_thread.join()
    returncode = proc.wait()
    if returncode != 0:
       raise RuntimeError(f'Process exited with code {returncode}')
finally:
    if proc.returncode is None:
        log.warning(f'Killing process {proc.pid}')
        proc.kill()

TL;DR: On Python 3.10

add '-u' to subprocess.Popen, found it in a forum either here or somewhere else
add '\n' to whatever string before encoding and sending through stdin.write()

I was trying to open a separate python file, and have the contents of that file show up in a tkinter text box.

the GUI is one I wanted to use with many programs, so it opens up the program in a subprocess. from searching many different forums I have gotten it to work and I am placing my answer here to hopefully help someone along the way.

My GUI starts a thread that calls the subprocess.

def select_program_thread(self):
    self.select_program_thrd = Thread(target=self.select_program, daemon=True)
    self.select_program_thrd.start()
    self.master.update()

def select_program(self):
    program_select = filedialog.askopenfilename(defaultextension='.py', initialdir='\path\to\your\dir')
    self.program = subprocess.Popen(['python','-u',program_select], # needed the '-u' to allow constant printing to the text box, found in a comment on a similar post, but don't remember which thread or person
                                    stdout=subprocess.PIPE,
                                    stderr=subprocess.STDOUT,
                                    stdin=subprocess.PIPE)
    while self.program.poll() == None: # check whether process is still running
        try:
            line = self.program.stdout.readline()
            if line:
                msg = line.decode() # incoming is bytes, need to decode
                self.print_to_text(msg.strip()) 
                self.program.stdout.flush() # unsure if needed but overall was working so left it alone
        except Exception as e:
            print(e)
        
     self.program.wait()

Then when my program needs any input from the console i have a button that grabs the string and transforms it into bytes

def console_input(self):
    try:
        txt = (self.console_text_input.get() + '\n') # Needed to add the '\n' to actually send it through. got stuck otherwise
        self.console_text_input.delete(0, END) # clear entry widget
        msg = txt.encode() # change to bytes
        self.program.stdin.write(msg) # write to subprocess
        self.program.stdin.flush() # clear stdin
    except Exception as e:
        print(e)
               

Test file

from time import sleep
print('I started Successfully')
x=3
while x>0:
    print(x)
    sleep(1)
    x-=1
print('Please send a command to the console')
v = input()
print(f'you typed {v}')
sleep(1)
print('Finished, Program Over')
sleep(1) # needed for stdout to be read

I needed at least one second at the end of the program to allow the stdout to be read and put into the text box, otherwise the process terminates before it is captured.

You can provide a file-like object to the stdin argument of subprocess.call().

The documentation for the Popen object applies here.

To capture the output, you should instead use subprocess.check_output(), which takes similar arguments. From the documentation:

>>> subprocess.check_output(
...     "ls non_existent_file; exit 0",
...     stderr=subprocess.STDOUT,
...     shell=True)
'ls: non_existent_file: No such file or directory\n'