title: dataframe select contiguous spans upon meeting condition
body: <p>Assume, I have the list of stopwords:</p>
  <pre><code>STOP = ['under', 'its', 'agreement', 'financed'] 
  </code></pre>
  <p>For the given dataframe:</p>
  <pre><code>lst = ['Kan.-based National', 'Kan.-based National Pizza', 'stock market', 
     'Pittsburg Kan.-based National Pizza', 'the stock market', 'revolving credit',
     'revolving credit agreement', 'its revolving credit agreement', 'under its revolving credit agreement', 
     'financed under its revolving credit agreement']
  
  df = pd.DataFrame(lst)
  </code></pre>
  <p>which is:</p>
  <pre><code>0   Kan.-based National
  1   Kan.-based National Pizza
  2   stock market
  3   Pittsburg Kan.-based National Pizza
  4   the stock market
  5   revolving credit
  6   revolving credit agreement
  7   its revolving credit agreement
  8   under its revolving credit agreement
  9   financed under its revolving credit agreement
  </code></pre>
  <p>I want to obtain:</p>
  <pre><code>out = ['Pittsburg Kan.-based National Pizza', 'the stock market', 'revolving credit',
         'revolving credit agreement', 'its revolving credit agreement', 'under its revolving credit agreement', 
         'financed under its revolving credit agreement']
  
  df_out = pd.DataFrame(out)
  </code></pre>
  <p>which is:</p>
  <pre><code>0   Pittsburg Kan.-based National Pizza
  1   the stock market
  2   revolving credit
  3   revolving credit agreement
  4   its revolving credit agreement
  5   under its revolving credit agreement
  6   financed under its revolving credit agreement
  </code></pre>
  <p><strong>Note:</strong> The order of rows isn't important.</p>
  <p><strong>Explanation:</strong></p>
  <p>Since, <code>'Kan.-based National'</code> and <code>'Kan.-based National Pizza'</code> differ by just one word <code>'Pizza'</code> and there are no words present from the <code>STOP</code> list, we want to choose the longest span, i.e. <code>'Kan.-based National Pizza'</code>.
  But, <code>'Pittsburg Kan.-based National Pizza'</code> also differs from <code>'Kan.-based National Pizza'</code> by just one word <code>'Pittsburg'</code> and there are no words present from the <code>STOP</code> list, we want to choose the longest span, i.e. <code>'Pittsburg Kan.-based National Pizza'</code>.</p>
  <p>We CANNOT choose <code>'financed under its revolving credit agreement'</code> as the longest span starting with <code>'revolving credit'</code> because the words are present in the <code>STOP</code> list. Hence, we will NOT delete it's smaller spans.</p>
  <p>Alternatively, on a side track, if the string starts with (a|an|the) and the difference between it's common spans is just one word. For eg - <code>&quot;stock market&quot;</code> and <code>&quot;the stock market&quot;</code>, we want to choose the longest span, i.e, <code>&quot;the stock market&quot;</code>.</p>
  <p>I tried to do:</p>
  <pre><code>delete_from_best_constituents = []
  for u in best_parse_constituents:
      for v in best_parse_constituents:
          if u.lower().startswith('the') or v.lower().startswith('the'):
              u_part = u.lower().split('the')[-1].strip()
              v_part =  v.lower().split('the')[-1].strip()
              cond1 = all([w.lower() not in STOP for w in u_part.split()])
              cond2 = all([w.lower() not in STOP for w in v_part.split()])
              if u_part == v.lower() or v_part == u.lower() and cond1 and cond2:
                  if not u.lower().startswith('the'):
                      delete_from_best_constituents.append(u)
  </code></pre>