title: Wrap an open stream with io.TextIOWrapper
body: <p>How can I wrap an open binary stream – a Python 2 <code>file</code>, a Python 3 <code>io.BufferedReader</code>, an <code>io.BytesIO</code> – in an <code>io.TextIOWrapper</code>?</p>
  
  <p>I'm trying to write code that will work unchanged:</p>
  
  <ul>
  <li>Running on Python 2.</li>
  <li>Running on Python 3.</li>
  <li>With binary streams generated from the standard library (i.e. I can't control what type they are)</li>
  <li>With binary streams made to be test doubles (i.e. no file handle, can't re-open).</li>
  <li>Producing an <code>io.TextIOWrapper</code> that wraps the specified stream.</li>
  </ul>
  
  <p>The <code>io.TextIOWrapper</code> is needed because its API is expected by other parts of the standard library. Other file-like types exist, but don't provide the right API.</p>
  
  <h1>Example</h1>
  
  <p>Wrapping the binary stream presented as the <code>subprocess.Popen.stdout</code> attribute:</p>
  
  <pre class="lang-python prettyprint-override"><code>import subprocess
  import io
  
  gnupg_subprocess = subprocess.Popen(
          ["gpg", "--version"], stdout=subprocess.PIPE)
  gnupg_stdout = io.TextIOWrapper(gnupg_subprocess.stdout, encoding="utf-8")
  </code></pre>
  
  <p>In unit tests, the stream is replaced with an <code>io.BytesIO</code> instance to control its content without touching any subprocesses or filesystems.</p>
  
  <pre class="lang-python prettyprint-override"><code>gnupg_subprocess.stdout = io.BytesIO("Lorem ipsum".encode("utf-8"))
  </code></pre>
  
  <p>That works fine on the streams created by Python 3's standard library. The same code, though, fails on streams generated by Python 2:</p>
  
  <pre class="lang-python prettyprint-override"><code>[Python 2]
  &gt;&gt;&gt; type(gnupg_subprocess.stdout)
  &lt;type 'file'&gt;
  &gt;&gt;&gt; gnupg_stdout = io.TextIOWrapper(gnupg_subprocess.stdout, encoding="utf-8")
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  AttributeError: 'file' object has no attribute 'readable'
  </code></pre>
  
  <h1>Not a solution: Special treatment for <code>file</code></h1>
  
  <p>An obvious response is to have a branch in the code which tests whether the stream actually is a Python 2 <code>file</code> object, and handle that differently from <code>io.*</code> objects.</p>
  
  <p>That's not an option for well-tested code, because it makes a branch that unit tests – which, in order to run as fast as possible, must not create any <em>real</em> filesystem objects – can't exercise.</p>
  
  <p>The unit tests will be providing test doubles, not real <code>file</code> objects. So creating a branch which won't be exercised by those test doubles is defeating the test suite.</p>
  
  <h1>Not a solution: <code>io.open</code></h1>
  
  <p>Some respondents suggest re-opening (e.g. with <code>io.open</code>) the underlying file handle:</p>
  
  <pre class="lang-python prettyprint-override"><code>gnupg_stdout = io.open(
          gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
  </code></pre>
  
  <p>That works on both Python 3 and Python 2:</p>
  
  <pre class="lang-python prettyprint-override"><code>[Python 3]
  &gt;&gt;&gt; type(gnupg_subprocess.stdout)
  &lt;class '_io.BufferedReader'&gt;
  &gt;&gt;&gt; gnupg_stdout = io.open(gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
  &gt;&gt;&gt; type(gnupg_stdout)
  &lt;class '_io.TextIOWrapper'&gt;
  </code></pre>
  
  <pre class="lang-python prettyprint-override"><code>[Python 2]
  &gt;&gt;&gt; type(gnupg_subprocess.stdout)
  &lt;type 'file'&gt;
  &gt;&gt;&gt; gnupg_stdout = io.open(gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
  &gt;&gt;&gt; type(gnupg_stdout)
  &lt;type '_io.TextIOWrapper'&gt;
  </code></pre>
  
  <p>But of course it <strong>relies on re-opening a real file</strong> from its file handle. So it fails in unit tests when the test double is an <code>io.BytesIO</code> instance:</p>
  
  <pre class="lang-python prettyprint-override"><code>&gt;&gt;&gt; gnupg_subprocess.stdout = io.BytesIO("Lorem ipsum".encode("utf-8"))
  &gt;&gt;&gt; type(gnupg_subprocess.stdout)
  &lt;type '_io.BytesIO'&gt;
  &gt;&gt;&gt; gnupg_stdout = io.open(gnupg_subprocess.stdout.fileno(), mode='r', encoding="utf-8")
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  io.UnsupportedOperation: fileno
  </code></pre>
  
  <h1>Not a solution: <code>codecs.getreader</code></h1>
  
  <p>The standard library also has the <code>codecs</code> module, which provides wrapper features:</p>
  
  <pre class="lang-python prettyprint-override"><code>import codecs
  
  gnupg_stdout = codecs.getreader("utf-8")(gnupg_subprocess.stdout)
  </code></pre>
  
  <p>That's good because it doesn't attempt to re-open the stream. But it fails to provide the <code>io.TextIOWrapper</code> API. Specifically, it <strong>doesn't inherit <code>io.IOBase</code></strong> and <strong>doesn't have the <code>encoding</code> attribute</strong>:</p>
  
  <pre class="lang-python prettyprint-override"><code>&gt;&gt;&gt; type(gnupg_subprocess.stdout)
  &lt;type 'file'&gt;
  &gt;&gt;&gt; gnupg_stdout = codecs.getreader("utf-8")(gnupg_subprocess.stdout)
  &gt;&gt;&gt; type(gnupg_stdout)
  &lt;type 'instance'&gt;
  &gt;&gt;&gt; isinstance(gnupg_stdout, io.IOBase)
  False
  &gt;&gt;&gt; gnupg_stdout.encoding
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
    File "/usr/lib/python2.7/codecs.py", line 643, in __getattr__
      return getattr(self.stream, name)
  AttributeError: '_io.BytesIO' object has no attribute 'encoding'
  </code></pre>
  
  <p>So <code>codecs</code> doesn't provide objects which substitute for <code>io.TextIOWrapper</code>.</p>
  
  <h1>What to do?</h1>
  
  <p>So how can I write code that works for both Python 2 and Python 3, with both the test doubles and the real objects, which <strong>wraps an <code>io.TextIOWrapper</code> around the already-open byte stream</strong>?</p>