title: Append data to HDF5 file with Pandas, Python
body: <p>I have large pandas DataFrames with financial data.
  I have no problem appending and concatenating additional columns and DataFrames to my .h5 file.</p>
  
  <p>The financial data is being updated every minute, I need to append a row of data to all of my existing tables inside of my .h5 file every minute.</p>
  
  <p>Here is what i have tried so far, but no matter what i do, it overwrites the .h5 file and does not just append data.</p>
  
  <p>HDFStore way:</p>
  
  <pre><code>#we open the hdf5 file
  save_hdf = HDFStore('test.h5') 
  
  ohlcv_candle.to_hdf('test.h5')
  
  #we give the dataframe a key value
  #format=table so we can append data
  save_hdf.put('name_of_frame',ohlcv_candle, format='table',  data_columns=True)
  
  #we print our dataframe by calling the hdf file with the key
  #just doing this as a test
  print(save_hdf['name_of_frame'])    
  </code></pre>
  
  <hr>
  
  <p>The other way I have tried it, to_hdf:   </p>
  
  <pre><code>#format=t so we can append data , mode=r+ to specify the file exists and
  #we want to append to it
  tohlcv_candle.to_hdf('test.h5',key='this_is_a_key', mode='r+', format='t')
  
  #again just printing to check if it worked 
  print(pd.read_hdf('test.h5', key='this_is_a_key'))
  </code></pre>
  
  <hr>
  
  <p>Here is what one of the DataFrames looks like after being read_hdf:</p>
  
  <pre><code>           time     open     high      low    close     volume           PP  
  0    1505305260  3137.89  3147.15  3121.17  3146.94   6.205397  3138.420000   
  1    1505305320  3146.86  3159.99  3130.00  3159.88   8.935962  3149.956667   
  2    1505305380  3159.96  3160.00  3159.37  3159.66   4.524017  3159.676667   
  3    1505305440  3159.66  3175.51  3151.08  3175.51   8.717610  3167.366667   
  4    1505305500  3175.25  3175.53  3170.44  3175.53   3.187453  3173.833333  
  </code></pre>
  
  <p>The next time I am getting data (every minute), i would like a row of it added to index 5 of all my columns..and then  6 and 7 ..and so on, without having to read and manipulate the entire file in memory as that would defeat the point of doing this.
  If there is a better way of solving this, do not be shy to recommend it.</p>
  
  <p>P.S. sorry for the formatting of that table in here</p>