title: Count word frequency of huge text file
body: <p>I have a huge text file (larger than the available RAM memory). I need to count the frequency of all words and output the word and the frequency count into a new file. The result should be sorted in the descending order of frequency count.</p>
  
  <p>My Approach:</p>
  
  <ol>
  <li>Sort the given file - external sort</li>
  <li>Count the frequency of each word sequentially, store the count in another file (along with the word)</li>
  <li>Sort the output file based of frequency count - external sort.</li>
  </ol>
  
  <p>I want to know if there are better approaches to do it. I have heard of disk based hash tables? or B+ trees, but never tried them before. </p>
  
  <p>Note: I have seen similar questions asked on SO, but none of them have to address the issue with data larger than memory.</p>
  
  <p>Edit: Based on the comments, agreed the a dictionary in practice should fit in the memory of today's computers. But lets take a hypothetical dictionary of words, that is huge enough not to fit in the memory.</p>