title: "surrogateescape" cannot escape certain characters
body: <p>Regarding reading and writing text files in Python, <a href="http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#unicode-basics" rel="noreferrer">one of the main Python contributors mentions</a> this regarding the <code>surrogateescape</code> Unicode Error Handler:</p>
  
  <blockquote>
    <p>[surrogateescape] handles decoding errors by squirreling the data away in a little used part of the Unicode code point space. When encoding, it translates those hidden away values back into the exact original byte sequence that failed to decode correctly.</p>
  </blockquote>
  
  <p>However, while opening a file and then attempting to write the output to another file:</p>
  
  <pre><code>input_file = open('someFile.txt', 'r', encoding="ascii", errors="surrogateescape")
  output_file = open('anotherFile.txt', 'w')
  
  for line in input_file:
      output_file.write(line)
  </code></pre>
  
  <p>Results in:</p>
  
  <pre><code>  File "./break-50000.py", line 37, in main
      output_file.write(line)
  UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc3' in position 3: surrogates not allowed
  </code></pre>
  
  <p>Note that the input file is <em>not</em> ASCII. However, it transverses hundreds of lines that contain non-ASCII characters just fine before it throws the exception on one particular line. The output file <em>must</em> be ASCII and loosing some characters is just fine.</p>
  
  <p>This is the line that is throwing the error when decoded as UTF-8:</p>
  
  <blockquote>
    <p>'Zoë\'s Coffee House'</p>
  </blockquote>
  
  <p>This is the hex encoding:</p>
  
  <pre><code>$ cat z.txt | hd
  00000000  27 5a 6f c3 ab 5c 27 73  20 43 6f 66 66 65 65 20  |'Zo..\'s Coffee |
  00000010  48 6f 75 73 65 27 0a                              |House'.|
  00000017
  </code></pre>
  
  <p><strong>Why might the <code>surrogateescape</code> Unicode Error Handler be returning a character that is <em>not</em> ASCII?</strong> This is with Python 3.2.3 on Kubuntu Linux 12.10. </p>