Dup2 And Pipe Shenanigans With Python And Windows

Say hello to bug.py: import os, sys stdout2 = os.dup(sys.stdout.fileno()) (r,w) = os.pipe() os.dup2(w,sys.stdout.fileno()) print('Hail Stan') z = os.read(r,1000) os.write(stdout2,

Solution 1:

I don't know if I've truly got to the bottomest bottom but I've mostly figured this one out.

sys.stdout is an object of type TextIOWrapper, whose sys.stdout.write() method eventually calls either os.write() or the C equivalent, on the filedescriptor sys.stdout.fileno(), which is 1. There are several types of filedescriptors: files, sockets, serial ports, terminals, pipes, etc... The C library functions write(), close(), etc... work on mostly any type of filedescriptor, but some features only work when the filedescriptor is of a suitable type. At creation time, the TextIOWrapper that is sys.stdout examines its filedescriptor (1) and determines that it's a terminal. On Windows only, sys.stdout.write() ends up doing some operations that are only valid if the filedescriptor 1 is truly a terminal. Once you os.dup2() so that 1 becomes an os.pipe(), and not a terminal, the Windows Python implementation crashes when it attempts to do terminal-specific operations on a pipe.

It's not clear to me that there's a way of causing sys.stdout to re-examine its filedescriptor so that it notices that it's not a terminal anymore, and avoid crashing the Python interpreter.

As a workaround, on Windows only, I'm doing stdout.write = lambda z: os.write(stdout.fileno(),z.encode() if hasattr(z,'encode') else z)

I haven't done a deep dive through the Python code base to see whether this is sufficient, but it does appear to allow my programs to run correctly, and print() no longer causes the Python interpreter to crash.

Many other people instead do sys.stdout = ..., but I did not want to do this for many reasons. For one, I was worried that some script or module might store a local cached copy of sys.stdout. Although it is possible to cache sys.stdout.write directly, I thought that this was less likely. Furthermore, it allows me to monkeypatch in my constructor, e.g. StreamCapture(sys.stdout), without having to refer to the global variable sys.stdout directly in my constructor (a "side-effect"). If you believe that a function is allowed to mutate its inputs, then this is side-effect-free.