Buffered and unbuffered stream redirections

Classification

Affected Components : builtin
Operating System : Linux
Python Versions : 2.6.x, 2.7.x, 3.1.x, 3.2.x
Reproducible : Yes

Source code

python -c 'print "x" * 8191' 1>&- ; echo $?

python -c 'print "x" * 8192' 1>&- ; echo $?

python -u -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?

python -u -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?

python -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?

python -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?

Steps to Produce/Reproduce

To reproduce the problem:

Open a terminal or system console
Copy a command line into the console, one at a time.
Press enter after copying the line to execute the command
Check the output
Repeat the steps for each command line to execute all the tests.

Description

Python behaviour in dealing with stream redirection changes in almost every version.

In python 2.5.x output redirection could be used as buffered or unbuffered *almost without problem.

Normally output streams are buffered and then when it is closed it's state is detected only when a flush is attempted.

Example:

# PYTHON 2.6

python -c 'print "x" * 8191' 1>&- ; echo $?
close failed in file object destructor:
Error in sys.excepthook:

Original exception was:
0

python -c 'print "x" * 8192' 1>&- ; echo $?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IOError: [Errno 9] Bad file descriptor
1

Python 2.6.x output redirection seems to fail to flush the output at all.

# PYTHON 2.6

# ===== TEST OUTPUT UNBUFFERED (option -u) =====
# expected to fail with code 1
python -u -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IOError: [Errno 9] Bad file descriptor
1

# expected to fail with code 1
# seems to be always unbuffered (option "-u" is ignored)
python -u -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?
1

# ===== TEST OUTPUT BUFFERED (without option -u) =====
# expected to fail with code 1
# PROBLEM => should fail with code 1 but is not failing
python -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
close failed in file object destructor:
Error in sys.excepthook:

Original exception was:
0

# expected to fail with code 1
python -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?
1

And the same happens in python 2.7.x as in following example.

# PYTHON 2.7

# ========= TEST OUTPUT UNBUFFERED (option -u) ===========
# expected to fail with code 1
python -u -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
IOError: [Errno 9] Bad file descriptor
1

# expected to fail with code 1
# seems to be always unbuffered (option "-u" is ignored)
python -u -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?
1

# ========= TEST OUTPUT BUFFERED (without option -u) ===========
# expected to fail with code 1
# PROBLEM => should fail with code 1 but is not failing
python -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
close failed in file object destructor:
Error in sys.excepthook:

Original exception was:
0

# expected to fail with code 1
python -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?
1

So far stderr is an exception as seems to be unbuffered (irrespective of the -u command option) in all python 2.x various.

And the behaviour changes again in newer releases as python 3.x stream operations all die with a SIGABRT which is also not correct both in term of operating system and software logic.

# PYTHON 3.x

# ========= TEST OUTPUT UNBUFFERED (option -u) ===========
# expected to fail with code 1
python3 -u -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Aborted
134

# expected to fail with code 1
# seems to be always unbuffered (option "-u" is ignored)
python3 -u -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?
Aborted
134

# ========= TEST OUTPUT BUFFERED (without option -u) ===========
# expected to fail with code 1
python3 -c 'import sys; print >> sys.stdout, "x"' 1>&- ; echo $?
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Aborted
134

# expected to fail with code 1
python3 -c 'import sys; print >> sys.stderr, "x"' 2>&- ; echo $?
Aborted
134

Python behaviour should reflect that state of the system in which the action is executed and consider conditions like: