title: Python pandas load csv ANSI Format as UTF-8
body: <p>I want to load a CSV File with pandas in Jupyter Notebooks which contains characters like ä,ö,ü,ß.</p>
  
  <p>When i open the csv file with Notepad++ here is one example row which causes trouble in ANSI Format:</p>
  
  <pre><code>Empf„nger;Empf„ngerStadt;Empf„ngerStraáe;Empf„ngerHausnr.;Empf„ngerPLZ;Empf„ngerLand
  </code></pre>
  
  <p>The correct UTF-8 outcome for Empf„nger should be: Empfänger</p>
  
  <p>Now when i load the CSV Data in Python 3.6 pandas on Windows with the following code:</p>
  
  <pre><code>df_a = pd.read_csv('file.csv',sep=';',encoding='utf-8')
  </code></pre>
  
  <p>I get and Error Message:</p>
  
  <pre><code>UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position xy: invalid continuation byte
  </code></pre>
  
  <p>Position 'xy' is  the position where the character occurs that causes the error message</p>
  
  <p>when i use the ansi format to load my csv file it works but display the umlaute incorrect.</p>
  
  <p>Example code:</p>
  
  <pre><code>df_a = pd.read_csv('afile.csv',sep=';',encoding='ANSI')
  </code></pre>
  
  <p>Empfänger is represented as:    Empf„nger</p>
  
  <p>Note: i have tried to convert the file to UTF-8 in Notepad++ and load it afterwards with the pandas module but i still get the same error.</p>
  
  <p>I have searched online for a solution but the provided solutions such as "change format in notepad++ to utf-8" or "use encoding='UTF-8'" or 'latin1' which gives me the same result as ANSI format or</p>
  
  <pre><code>import chardet
  
  with open('afile.csv', 'rb') as f:
      result = chardet.detect(f.readline())
  
  df_a = pd.read_csv('afile.csv',sep=';',encoding=result['encoding'])
  </code></pre>
  
  <p>didnt work for me.</p>
  
  <pre><code>encoding='cp1252'
  </code></pre>
  
  <p>throws the following exception: </p>
  
  <pre><code>UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 2: character maps to &lt;undefined&gt;
  </code></pre>
  
  <p>I also tried to replace Strings afterwards with the <code>x.replace()</code> method but the character ü disappears completely after loaded into a pandas DataFrame</p>