title: Best way to simulate "group by" from bash?
body: <p>Suppose you have a file that contains IP addresses, one address in each line:</p>
  
  <pre><code>10.0.10.1
  10.0.10.1
  10.0.10.3
  10.0.10.2
  10.0.10.1
  </code></pre>
  
  <p>You need a shell script that counts for each IP address how many times it appears in the file. For the previous input you need the following output:</p>
  
  <pre><code>10.0.10.1 3
  10.0.10.2 1
  10.0.10.3 1
  </code></pre>
  
  <p>One way to do this is:</p>
  
  <pre><code>cat ip_addresses |uniq |while read ip
  do
      echo -n $ip" "
      grep -c $ip ip_addresses
  done
  </code></pre>
  
  <p>However it is really far from being efficient.</p>
  
  <p>How would you solve this problem more efficiently using bash?</p>
  
  <p>(One thing to add: I know it can be solved from perl or awk, I'm interested in a better solution in bash, not in those languages.)</p>
  
  <p>ADDITIONAL INFO:</p>
  
  <p>Suppose that the source file is 5GB and the machine running the algorithm has 4GB. So sort is not an efficient solution, neither is reading the file more than once.</p>
  
  <p>I liked the hashtable-like solution - anybody can provide improvements to that solution?</p>
  
  <p>ADDITIONAL INFO #2:</p>
  
  <p>Some people asked why would I bother doing it in bash when it is way easier in e.g. perl. The reason is that on the machine I had to do this perl wasn't available for me. It was a custom built linux machine without most of the tools I'm used to. And I think it was an interesting problem.</p>
  
  <p>So please, don't blame the question, just ignore it if you don't like it. :-)</p>