[ACCEPTED]-Python: How to read stdout non blocking from another process?-popen

Accepted answer

Score: 12

What is happening is buffering on the writer 11 side. Since you are writing such small chunks 10 from the little code snippet the underlying 9 FILE object is buffering the output until 8 the end. The following works as you expect.

#!/usr/bin/python

import sys
import subprocess

p = subprocess.Popen("""python -c '
from time import sleep ; import sys
for i in range(3):
    sleep(1)
    print "Hello", i
    sys.stdout.flush()
'""", shell = True, stdout = subprocess.PIPE)

while True:
    inline = p.stdout.readline()
    if not inline:
        break
    sys.stdout.write(inline)
    sys.stdout.flush()

print "Done"

However, you 7 may not be expecting the right thing. The 6 buffering is there to reduce the number 5 of system calls in order to make the system 4 more efficient. Does it really matter to 3 you that the whole text is buffered until 2 the end before you write it to a file? Don't 1 you still get all the output in the file?

Score: 5

the following code would print stdout line 3 by line as the subprocess runs until the 2 readline() method returns an empty string:

p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
for line in iter(p.stdout.readline, ''):
    print line
p.stdout.close()
print 'Done'

update 1 relating to your question better:

import subprocess

p = subprocess.Popen(['python'], stdout=subprocess.PIPE, stdin=subprocess.PIPE)
p.stdin.write("""
from time import sleep ; import sys
for i in range(3):
    sleep(1)
    print "Hello", i
    sys.stdout.flush()
""")
p.stdin.close()
for line in iter(p.stdout.readline, ''):
    print line
p.stdout.close()
print 'Done'

Source: stackoverflow.com

[ACCEPTED]-Python: How to read stdout non blocking from another process?-popen

Why does shell=True eat my subprocess.Popen stdout?

Subprocess.Popen: cloning stdout and stderr both to terminal and variables

How to get environment from a subprocess?

pipe large amount of data to stdin while using subprocess.Popen

Launch a totally independent process from Python

Send Ctrl-C to remote processes started via subprocess.Popen and ssh

python: raise child_exception, OSError: [Errno 2] No such file or directory

Can I pipe a io.BytesIO() stream to subprocess.popen() in Python?

Python Popen sending to process on stdin, receiving on stdout

Python subprocess Popen.communicate() equivalent to Popen.stdout.read()?