title: Encoding two different languages using pd.read_csv problem
body: <p>I am building a Neural Machine Automatic Translator from German to Arabic. I am reading a CSV file containing German sentences and their corresponding Arabic translations. I want to read both languages at the same time using <code>pd.read_csv</code>. I have tried all the codes for all languages in this <a href="https://docs.python.org/3/library/codecs.html#standard-encodings" rel="nofollow noreferrer">Python documentation</a> but none of them worked.</p>
  <p><a href="https://i.stack.imgur.com/2JBdP.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2JBdP.png" alt="Aliases encoding for all languages" /></a></p>
  <p>The only thing that worked best for me is this:</p>
  <pre><code>df = pd.read_csv(&quot;DLATS.csv&quot;, encoding ='windows-1256')
  </code></pre>
  <p>'windows-1256' is the encoding Alias for the Arabic language. But the problem is that it doesn't catch the German special characters like (ä) but it converts them into question marks (?). So the word drängte became dr?ngte.</p>
  <p>So, can anyone please help me to solve this problem or how to work around it? I have thought of separating the German and Arabic sentences in separate CSV files so that each CSV file contains one row only, and then maybe I will try to mix them in the Python code. But it seems that <code>pd.read_csv</code> requires at least two columns in the CSV file to work.</p>
  <p>Update: I have noticed that the original csv file contains these problems as well for the German language. So, I have finally managed to solve my problem by reading excel directly instead of csv since the original file is in Excel, so I used pd.read_excel without any encoding attribute and it worked well. I didn't know before that pandas has pd.read_excel.</p>