title: Shell: script to group strings by substring
body: <p>I have a program (sorry changing this is not an option) that is outputting log files with upwards of 500k lines.</p>
  
  <p>I am trying to group together lines (and then sort these groups) in the log file based on a substring with in the lines</p>
  
  <p>For example I have lines similar to below:</p>
  
  <pre><code>SELECT something WHERE TIM BETWEEN '*' AND '*' AND something;
  </code></pre>
  
  <p>what im looking to group on is the <code>TIM BETWEEN '*' AND '*'</code> where * matches between lines for example:</p>
  
  <pre><code>SELECT something WHERE TIM BETWEEN '2010-03-04' AND '2010-03-10' AND something;
  SELECT something WHERE TIM BETWEEN '2011-01-28' AND '2011-02-05' AND something;
  SELECT something WHERE TIM BETWEEN '2010-03-04' AND '2010-03-10' AND something;
  SELECT something WHERE TIM BETWEEN '2011-01-28' AND '2011-02-05' AND something;
  </code></pre>
  
  <p>would be grouped as such in the output:</p>
  
  <pre><code>SELECT something WHERE TIM BETWEEN '2010-03-04' AND '2010-03-10' AND something;
  SELECT something WHERE TIM BETWEEN '2010-03-04' AND '2010-03-10' AND something;
  SELECT something WHERE TIM BETWEEN '2011-01-28' AND '2011-02-05' AND something;
  SELECT something WHERE TIM BETWEEN '2011-01-28' AND '2011-02-05' AND something;
  </code></pre>
  
  <p>with each group also having been sorted based on the whole string so where the "somethings" are similar the are next to each other?</p>
  
  <p>I have been trying to put a shell script together to output what i want reading from a log file but haven't had any success!</p>
  
  <p>Edit: I need to also mention that 'something' can be multiple words for example:</p>
  
  <pre><code>SELECT blah1, blah2 or SELECT blah1, blah2, blah3
  </code></pre>