title: Python: Traceback codecs.charmap_decode(input,self.errors,decoding_table)[0]
body: <p>Following is sample code, aim is just to merges text files from give folder and it's sub folder. i am getting Traceback occasionally so not sure where to look. also need some help to enhance the code to prevent blank line being merge &amp; to display no lines in merged/master file. Probably it's good idea to before merging file, some cleanup should performed or just to ignores blank line during merging process. </p>
  
  <p>Text file in folder is not more then 1000 lines but aggregate master file could cross 10000+ lines very easily.</p>
  
  <pre><code>import os
  root = 'C:\\Dropbox\\ans7i\\'
  files = [(path,f) for path,_,file_list in os.walk(root) for f in file_list]
  out_file = open('C:\\Dropbox\\Python\\master.txt','w')
  for path,f_name in files:
      in_file = open('%s/%s'%(path,f_name), 'r')
  
      # write out root/path/to/file (space) file_contents
      for line in in_file:
          out_file.write('%s/%s %s'%(path,f_name,line))
      in_file.close()
  
      # enter new line after each file
      out_file.write('\n')
  
  with open('master.txt', 'r') as f:
    lines = f.readlines()
  with open('master.txt', 'w') as f:
    f.write("".join(L for L in lines if L.strip())) 
  
  
  
  Traceback (most recent call last):
    File "C:\Dropbox\Python\master.py", line 9, in &lt;module&gt; for line in in_file:
    File "C:\PYTHON32\LIB\encodings\cp1252.py", line  23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0]  
  UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 972: character maps to &lt;undefined&gt;  
  </code></pre>