title: python: custom pandas.DataFrame to dictionary function: some entries are lost
body: <p>I want to read a <code>.xlsx</code> file, do some things with the data and convert it to a <code>dict</code> to save it in a <code>.json</code> file. To do that I use <code>Python3</code> and <code>pandas</code>.</p>
  <p>This is the code:</p>
  <pre><code>import pandas as pd
  import json
  
  xls = pd.read_excel(
      io = &quot;20codmun.xlsx&quot;,
      converters = {
          &quot;CODAUTO&quot; : str,
          &quot;CPRO&quot; : str,
          &quot;CMUN&quot; : str,
          &quot;DC&quot; : str
          }
      )
  
  print(xls)
  #print(xls.columns.values)
  
  outDict = {}
  
  print(len(xls[&quot;NOMBRE&quot;])) # 8131 rows
  
  for i in range(len(xls.index)):
      
      codauto = xls[&quot;CODAUTO&quot;][i]
      cpro = xls[&quot;CPRO&quot;][i]
      cmun = xls[&quot;CMUN&quot;][i]
      dc = xls[&quot;DC&quot;][i]
      aemetId = cpro + cmun
      
      outDict[xls[&quot;NOMBRE&quot;][i]] = {
          &quot;CODAUTO&quot; : codauto,
          &quot;CPRO&quot; : cpro,
          &quot;CMUN&quot; : cmun,
          &quot;DC&quot; : dc,
          &quot;AEMET_ID&quot; : aemetId
          }
  
  print(i) # 8130
  print(len(outDict)) # 8114 entries, SOME ENTIRES ARE LOST!!!!!
  
  #print(outDict[&quot;Petrer&quot;])
  
  with open(&quot;data.json&quot;, &quot;w&quot;) as outFile:
      json.dump(outDict, outFile)
  </code></pre>
  <p>I add <a href="https://www.ine.es/daco/daco42/codmun/codmunmapa.htm" rel="nofollow noreferrer">here</a> the source of the <code>.xlsx</code> file (Spanish government). Select &quot;Fichero con todas las provincias&quot;. You have to delete the first row.</p>
  <p>As you can see, the <code>pandas.DataFrame</code> has 8131 rows, the <code>for</code> index at the end is 8130, but the length of the final <code>dict</code> is 8114, so some data is lost!</p>
  <p>You can check that &quot;Aljucén&quot; is on the <code>.xlsx</code> file, but not in the <code>.json</code> one. Edit: Solved using <code>json.dump(outDict, outFile, ensure_ascii=False)</code></p>