title: Encoding Issues when reading .htm files with Python
body: <p>I am attempting to read in a large set of <code>.htm</code> files with Python. To do so I am using the following:</p>
  
  <pre><code>HtmlFile = codecs.open(file, 'r')
  text = BeautifulSoup(HtmlFile.read()).text
  </code></pre>
  
  <p>However, this results in the following error:</p>
  
  <pre><code>UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 411: 
  character maps to &lt;undefined&gt;
  </code></pre>
  
  <p>So, I tried encoding with <code>utf-8</code> like so:</p>
  
  <pre><code>HtmlFile = codecs.open(file, 'r', encoding='utf-8')
  text = BeautifulSoup(HtmlFile.read()).text
  </code></pre>
  
  <p>And then I got this error:</p>
  
  <pre><code>UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 4565: 
  invalid start byte
  </code></pre>
  
  <p>I tried following the advice <a href="https://stackoverflow.com/questions/22216076/unicodedecodeerror-utf8-codec-cant-decode-byte-0xa5-in-position-0-invalid-s">here</a>, but it was not helping. Any help would be greatly appreciated! </p>