title: How to use unidecode in python (3.3)
body: <p>I'm trying to remove all non-ascii characters from a text document. I found a package that should do just that, <a href="https://pypi.python.org/pypi/Unidecode">https://pypi.python.org/pypi/Unidecode</a> </p>
  
  <p>It should accept a string and convert all non-ascii characters to the closest ascii character available. I used this same module in perl easily enough by just calling <code>while (&lt;input&gt;) { $_ = unidecode($_); }</code> and this one is a direct port of the perl module, the documentation indicates that it should work the same.</p>
  
  <p>I'm sure this is something simple, I just don't understand enough about character and file encoding to know what the problem is. My origfile is encoded in UTF-8 (converted from UCS-2LE). The problem may have more to do with my lack of encoding knowledge and handling strings wrong than the module, hopefully someone can explain why though. I've tried everything I know without just randomly inserting code and search the errors I'm getting with no luck so far.</p>
  
  <p>Here's my python</p>
  
  <pre><code>from unidecode import unidecode
  
  def toascii():
      origfile = open(r'C:\log.convert', 'rb')
      convertfile = open(r'C:\log.toascii', 'wb')
  
      for line in origfile:
          line = unidecode(line)
          convertfile.write(line)
  
      origfile.close()
      convertfile.close()
  
  toascii();
  </code></pre>
  
  <p>If I don't open the original file in byte mode (<code>origfile = open('file.txt','r'</code>) then I get an error <code>UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1563: character maps to &lt;undefined&gt;</code> from the <code>for line in origfile:</code> line. </p>
  
  <p>If I do open it in byte mode <code>'rb'</code> I get <code>TypeError: ord() expected string length 1, but int found</code> from the <code>line = unidecode(line)</code> line.</p>
  
  <p>if I declare line as a string <code>line = unidecode(str(line))</code> then it will write to the file, but... not correctly. <code>\r\n'b'\xef\xbb\xbf[ 2013.10.05 16:18:01 ] User_Name &gt; .\xe2\x95\x90\xe2\x95\x90\xe2\x95\x90\xe2\x95\x90\</code> It's writing out the \n, \r, etc and unicode characters instead of converting them to anything.</p>
  
  <p>If I convert the line to string as above, and open the convertfile in byte mode <code>'wb'</code> it gives the error <code>TypeError: 'str' does not support the buffer interface</code></p>
  
  <p>If I open it in byte mode without declaring it a string <code>'wb'</code> and <code>unidecode(line)</code> then I get the <code>TypeError: ord() expected string length 1, but int found</code> error again.</p>