title: read subprocess output multi-byte characters one by one
body: <p>I'm running a process using subprocess:</p>
  
  <pre><code>    p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
  </code></pre>
  
  <p>What I want to do is to read the output characters one by one in a loop:</p>
  
  <pre><code>while something:
      char = p.stdout.read(1)
  </code></pre>
  
  <p>In python3 <code>subprocess.Popen().stdout.read()</code> returns <code>bytes()</code> not <code>str()</code>. I want to use it as str so I have to do:</p>
  
  <pre><code>    char = char.decode("utf-8")
  </code></pre>
  
  <p>and it works fine with ascii characters.</p>
  
  <p>But with non-ascii characters (eg. greek letters) I get a UnicodeDecodeError. That's why greek chars are consisted of more than one bytes. Here's the problem:</p>
  
  <pre><code>&gt;&gt;&gt; b'\xce\xb5'.decode('utf-8')
  'ε'
  &gt;&gt;&gt; b'\xce'.decode('utf-8') # b'\xce' is what subprocess...read(1) returns - one byte
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xce in position 0: unexpected end of data
  &gt;&gt;&gt; 
  </code></pre>
  
  <p>How can I deal with this?
  The output of <code>subprocess.Popen().stdout.read()</code> (as a string) can be something like "lorem ipsum εφδφδσloremipsum".</p>
  
  <p>I want to read one character a time but this character can consisted of multiple bytes.</p>