title: Most efficient way to store data on drive
body: <p>baseline - I have CSV data with 10,000 entries. I save this as 1 csv file and load it all at once. </p>
  
  <p>alternative - I have CSV data with 10,000 entries. I save this as 10,000 CSV files and load it individually. </p>
  
  <p>Approximately how much more inefficient is this computationally. I'm not hugely interested in memory concerns. The purpose of the alternative method is because I frequently need to access subsets of the data and don't want to have to read the entire array. </p>
  
  <p>I'm using python. </p>
  
  <p>Edit: I can other file formats if needed. </p>
  
  <p>Edit1: SQLite wins. Amazingly easy and efficient compared to what I was doing before. </p>