title: UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 7240: character maps to <undefined>
body: <p>I am student doing my master thesis. As part of my thesis, I am working with <strong>python</strong>. I am reading a log file of <code>.csv</code> format and writing the extracted data to another <code>.csv</code> file in a well formatted way. However, when the file is read, I am getting this error:</p>
  
  <blockquote>
    <p>Traceback (most recent call last):   File
    "C:\Users\SGADI\workspace\DAB_Trace\my_code\trace_parcer.py", line 19,
    in  for row in reader:</p>
    
    <ul>
    <li>File "C:\Users\SGADI\Desktop\Python-32bit-3.4.3.2\python-3.4.3\lib\encodings\cp1252.py",
    line 23, in decode return <code>codecs.charmap_decode(input,self.errors,decoding_table)[0]</code></li>
    <li>UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 7240: character maps to <code>&lt;undefined&gt;</code></li>
    </ul>
  </blockquote>
  
  <pre><code>import csv
  import re
  #import matplotlib
  #import matplotlib.pyplot as plt
  import datetime
  #import pandas
  #from dateutil.parser import parse
  #def parse_csv_file():
  timestamp = datetime.datetime.strptime('00:00:00.000', '%H:%M:%S.%f')
  timestamp_list = []
  snr_list = []
  freq_list = []
  rssi_list = []
  dab_present_list = []
  counter = 0
  f =  open("output.txt","w")
  with open('test_log_20150325_gps.csv') as csvfile:
      reader = csv.reader(csvfile, delimiter=';') 
      for row in reader:
          #timestamp = datetime.datetime.strptime(row[0], '%M:%S.%f')
          #timestamp.split(" ",1)
  
          timestamp = row[0]
          timestamp_list.append(timestamp)
  
  
          #timestamp = row[0]
          details = row[-1]
          counter += 1
          print (counter)
          #if(counter &gt; 25000):
          #  break
          #timestamp = datetime.datetime.strptime(row[0], '%M:%S.%f')  
  
  
  
          #timestamp_list.append(float(timestamp))
  
          #search for SNRLevel=\d+
          snr = re.findall('SNRLevel=(\d+)', details)
          if snr == []:
              snr = 0
          else:
              snr = snr[0]
          snr_list.append(int(snr))
  
          #search for Frequency=09ABC
          freq = re.findall('Frequency=([0-9a-fA-F]+)', details)
          if freq == []:
              freq = 0
          else:
              freq = int(freq[0], 16)
          freq_list.append(int(freq))
  
          #search for RSSI=\d+
          rssi = re.findall('RSSI=(\d+)', details)
          if rssi == []:
              rssi = 0
          else:
              rssi = rssi[0]
          rssi_list.append(int(rssi))
  
          #search for DABSignalPresent=\d+
          dab_present = re.findall('DABSignalPresent=(\d+)', details)
          if dab_present== []:
              dab_present = 0
          else:
              dab_present = dab_present[0]
          dab_present_list.append(int(dab_present))
  
          f.write(str(timestamp) + "\t")
          f.write(str(freq) + "\t")
          f.write(str(snr) + "\t")
          f.write(str(rssi) + "\t")
          f.write(str(dab_present) + "\n")
          print (timestamp, freq, snr, rssi, dab_present)
  
          #print (index+1)
  
          #print(timestamp,freq,snr)
          #print (counter)
  #print(timestamp_list,freq_list,snr_list,rssi_list)
  
  
  '''if  snr != []:
             if freq != []:
                 timestamp_list.append(timestamp)
                 snr_list.append(snr)
                 freq_list.append(freq)
  f.write(str(timestamp_list) + "\t")
  f.write(str(freq_list) + "\t")
  f.write(str(snr_list) + "\n")
  
  print(timestamp_list,freq_list,snr_list)'''
  f.close()
  </code></pre>
  
  <p>I searched for the special character and I did not find any. I searched the Internet which suggested to change the format: I tried ut8, latin1 and few other formats, but i am still getting this error. Can you please help me how to solve with <code>pandas</code> as well. I also tried with <code>pandas</code> but I am still getting the error.
  I even removed a line in the log file, but the error occurs in the next line.</p>
  
  <p>Please help me finding a solution, thank you.</p>