title: Improve Row Append Performance On Pandas DataFrames
body: <p>I am running a basic script that loops over a nested dictionary, grabs data from each record, and appends it to a Pandas DataFrame. The data looks something like this:</p>
  
  <pre class="lang-py prettyprint-override"><code>data = {"SomeCity": {"Date1": {record1, record2, record3, ...}, "Date2": {}, ...}, ...}
  </code></pre>
  
  <p>In total it has a few million records. The script itself looks like this:</p>
  
  <pre class="lang-py prettyprint-override"><code>city = ["SomeCity"]
  df = DataFrame({}, columns=['Date', 'HouseID', 'Price'])
  for city in cities:
      for dateRun in data[city]:
          for record in data[city][dateRun]:
              recSeries = Series([record['Timestamp'], 
                                  record['Id'], 
                                  record['Price']],
                                  index = ['Date', 'HouseID', 'Price'])
              FredDF = FredDF.append(recSeries, ignore_index=True)
  </code></pre>
  
  <p>This runs painfully slow, however. Before I look for a way to parallelize it, I just want to make sure I'm not missing something obvious that would make this perform faster as it is, as I'm still quite new to Pandas.</p>