title: Iteratively parsing a JSON file with ijson
body: <p>I want to parse a huge json file in chunks. I want to use chunks of it without loading the whole thing. 
  The data can be found here 
  <a href="http://jmcauley.ucsd.edu/data/amazon/" rel="nofollow noreferrer">http://jmcauley.ucsd.edu/data/amazon/</a></p>
  
  <p>When I use ijson to do that I get an error of <code>JSONError: Additional data</code>.
  Is there any way to do this?</p>
  
  <p>My code:##produces Additional Data Error</p>
  
  <pre><code>import pandas as pd
  
  
  file = open('Books_5.json',"r") ##Books_5.json is the 5-core small dataset
  objects = ijson.items(file, 'meta.data.item')
  reviews = (o for o in objects if o['type']=='reviewText')
  for review in reviews : print(review)
  </code></pre>
  
  <p>This does work, but is very slow:</p>
  
  <pre><code>path='Books_5.json'
  def parse(path):
    g = open(path, 'rb')
    for l in g:
      yield eval(l)
  def getDF(path):
    i = 0
    df = {}
    for d in parse(path):
      df[i] = d
      i += 1
    return pd.DataFrame.from_dict(df, orient='index')
  
  df = getDF(path)
  df.info()
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/NGaet.png" rel="nofollow noreferrer">result of the code</a></p>