title: How to find unique values in a large JSON file?
body: <p>I've 2 json files of size <code>data_large(150.1mb)</code> and <code>data_small(7.5kb)</code>. The content inside each file is of type <code>[{"score": 68},{"score": 78}]</code>. I need to find the list of unique scores from each file.</p>
  
  <p>While dealing with <strong>data_small</strong>, I did the following and I was able to view its content with <code>0.1 secs</code>.</p>
  
  <pre><code>with open('data_small') as f:
      content = json.load(f)
  
  print content # I'll be applying the logic to find the unique values later.
  </code></pre>
  
  <p>But while dealing with <strong>data_large</strong>, I did the following and my system got hanged, slow and had to force shut-it down to bring it into its normal speed. It took around <code>2 mins</code>  to print its content.</p>
  
  <pre><code>with open('data_large') as f:
      content = json.load(f)
  
  print content # I'll be applying the logic to find the unique values later.
  </code></pre>
  
  <p><strong>How can I increase the efficiency of the program while dealing with large data-sets?</strong></p>