title: convert em-dash to hyphen in python
body: <p>I'm converting csv files into python Dataframe. And in the original file, one of the column has characters em-dash. I want it replaced by hyphen "-".</p>
  
  <p>Partial original file from csv:</p>
  
  <pre><code> NoDemande     NoUsager     Sens    IdVehicule     NoConduteur     HeureDebutTrajet    HeureArriveeSurSite    HeureEffective'
  42192001801   42192002715    —        157Véh       42192000153    ...
  42192000003   42192002021    +        157Véh       42192000002    ...
  42192001833   42192000485    —      324My3FVéh     42192000157    ...
  </code></pre>
  
  <p>My code:</p>
  
  <pre><code>#coding=latin-1
  import pandas as pd
  import glob
  
  pd.set_option('expand_frame_repr', False)
  
  path = r'D:\Python27\mypfe\data_test'
  allFiles = glob.glob(path + "/*.csv")
  frame = pd.DataFrame()
  list_ = []
  for file_ in allFiles:
      df = pd.read_csv(file_,index_col=None,header=0,sep=';',parse_dates=['HeureDebutTrajet','HeureArriveeSurSite','HeureEffective'],
                        dayfirst=True)
      df['Sens'].replace(u"\u2014","-",inplace=True,regex=True)
      list_.append(df)
  </code></pre>
  
  <p>And it doesn't work at all, every time it only convert them into <code>?</code>, which looks like it:</p>
  
  <pre><code>42191001122  42191002244    ?            181Véh   42191000114  ...
  42191001293  42191001203    ?         319M9pVéh   42191000125  ...
  42191000700  42191000272    ?            183Véh   42191000072  ...
  </code></pre>
  
  <p>And because I have french characters in the file, I'm using <code>latin-1</code> instead of <code>utf-8</code>. If I delete the first line and write like this:</p>
  
  <pre><code>df = pd.read_csv(file_,index_col=None,header=0,sep=';',encoding='windows-1252',parse_dates=['HeureDebutTrajet','HeureArriveeSurSite','HeureEffective'],
                            dayfirst=True)
  </code></pre>
  
  <p>The result will be:</p>
  
  <pre><code>42191001122  42191002244  â??           181VÃ©h   42191000114   ...
  42191001293  42191001203  â??        319M9pVÃ©h   42191000125   ...
  42191000700  42191000272  â??           183VÃ©h   42191000072   ...
  </code></pre>
  
  <p>How can I make all the em-dash <code>—</code> replaced by <code>-</code>?</p>
  
  <p>I added the part about <code>repr</code>:</p>
  
  <pre><code>for line in open(file_):
      print repr(line)
  </code></pre>
  
  <p>And the result turns out:</p>
  
  <pre><code>'"42191002384";"42191000118";"\xe2\x80\x94";"";"42191000182";...
  '"42191002464";"42191001671";"+";"";"42191000182";...
  '"42191000045";"42191000176";"\xe2\x80\x94";"620M9pV\xc3\xa9h";"42191000003";...
  '"42191001305";"42191000823";"\xe2\x80\x94";"310V7pV\xc3\xa9h";"42191000126";...
  </code></pre>