[ACCEPTED]-Bypassing buffering of subprocess output with popen in C or Python-pipe

Accepted answer
Score: 16

In general, the standard C runtime library 25 (that's running on behalf of just about 24 every program on every system, more or less;-) detects 23 whether stdout is a terminal or not; if 22 not, it buffers the output (which can be 21 a huge efficiency win, compared to unbuffered 20 output).

If you're in control of the program 19 that's doing the writing, you can (as another 18 answer suggested) flush stdout continuously, or 17 (more elegantly if feasible) try to force 16 stdout to be unbuffered, e.g. by running 15 Python with the -u commandline flag:

-u     : unbuffered binary stdout and stderr (also PYTHONUNBUFFERED=x)
         see man page for details on internal buffering relating to '-u'

(what 14 the man page adds is a mention of stdin 13 and issues with binary mode[s]).

If you can't 12 or don't want to touch the program that's 11 writing, -u or the like on the program that's 10 just reading is unlikely to help (the buffering 9 that matters most is the one happening on 8 the writer's stdout, not the one on the 7 reader's stdin). The alternative is to trick 6 the writer into believing that it's writing 5 to a terminal (even though in fact it's 4 writing to another program!), via the pty standard 3 library module or the higher-level third 2 party pexpect module (or, for Windows, its port 1 wexpect).

Score: 1

Thats correct, and applies to both Windows 4 and Linux (and possibly other systems), with 3 popen() and fopen(). If you want the output buffer to 2 be dispatched before 4096 bytes, use fflush() (on 1 C) or sys.stdout.flush() (Python).

More Related questions