title: Pandas filtering on max range
body: <p>I'm working on text mining problem and using Pandas for text processing. From the following example I need to pick only those row which have the max span (<code>start</code> - <code>end</code>) within the same category (<code>cat</code>)</p>
  
  <p>Given this dataframe:</p>
  
  <pre><code>                             name  start  end        cat
  0                        coumadin      0    8       DRUG
  1                       albuterol     18   27       DRUG
  2               albuterol sulfate     18   35       DRUG
  3                         sulfate     28   35       DRUG
  4                             2.5     36   39   STRENGTH
  5                          2.5 mg     36   42   STRENGTH
  6                    2.5 mg /3 ml     36   48   STRENGTH
  7                           0.083     50   55   STRENGTH
  8                         0.083 %     50   57   STRENGTH
  9          2.5 mg /3 ml (0.083 %)     36   58   STRENGTH
  10                       solution     59   67       FORM
  11      solution for nebulization     59   84       FORM
  12                   nebulization     72   84      ROUTE
  13                        one (1)     90   97  FREQUENCY
  14                            neb     98  101      ROUTE
  15                 neb inhalation     98  112      ROUTE
  16                     inhalation    102  112      ROUTE
  17                            q4h    113  116  FREQUENCY
  18                          every    118  123  FREQUENCY
  19                  every 4 hours    118  131  FREQUENCY
  20            q4h (every 4 hours)    113  132  FREQUENCY
  21  q4h (every 4 hours) as needed    113  142  FREQUENCY
  22                      as needed    133  142  FREQUENCY
  23                        dyspnea    147  154     REASON
  </code></pre>
  
  <p>I need to get the following:</p>
  
  <pre><code>            name           start   end       cat
  0                coumadin      0    8       DRUG
  2   albuterol sulfate         18    35      DRUG
  9   2.5 mg /3 ml (0.083 %)    36    58  STRENGTH
  11  solution for nebulization 59    84      FORM
  13                one (1)     90    97  FREQUENCY
  15  neb inhalation            98    112    ROUTE
  21  q4h (every 4 hours) as needed   113 142 FREQUENCY
  23  dyspnea                  147    154   REASON
  </code></pre>
  
  <p>What I tried is to <code>groupby</code> by the category and then compute the max difference (<code>end</code>-<code>start</code>). However I got stuck how to find the max span between for the same entity within the category. I guess it should not be very tricky</p>
  
  <p><strong>COMMENT</strong></p>
  
  <p>Thank you all for suggestions, but I need ALL possible entities within each category. For example, in DRUG, there are two relevant drugs: <code>coumadin</code> and <code>albuterol sulfate</code>, and some fractions of them (<code>albuterol</code> and <code>sulfate</code>). I need to remove only (<code>albuterol</code> and <code>sulfate</code>) while keeping <code>coumadin</code> and <code>albuterol sulfate</code>. The same logic for other categories.</p>
  
  <p>For example, rows 4-8 are all bits of a complete row 9, thus I need to keep only row 9. Rows 1 and 3 are parts of the row 2, thus I need to keep row 2 (in addition to row 0). Etc.</p>
  
  <p>Obviously, all constituents ('bits') are within the max range, but the problem is to find the max (or unifying range) of the same entity and its constituents)</p>
  
  <p><strong>COMMENT 2</strong></p>
  
  <p>A possible solution could be: to find all overlapping intervals within the same category <code>cat</code> and pick the largest. I'm trying to implement, but not luck so far.</p>
  
  <p><strong>Possible Solution</strong></p>
  
  <p>I sorted columns by ascending and descending order:</p>
  
  <p>df.sort_values(by=[1,2], ascending=[True, False])</p>
  
  <pre><code>                           0           1     2     3
  0   coumadin                           0     8  DRUG
  2   albuterol sulfate                 18    35  DRUG
  1   albuterol                         18    27  DRUG
  3   sulfate                           28    35  DRUG
  9   2.5 mg /3 ml (0.083 %)            36    58  STRENGTH
  6   2.5 mg /3 ml                      36    48  STRENGTH
  5   2.5 mg                            36    42  STRENGTH
  4   2.5                               36    39  STRENGTH
  8   0.083 %                           50    57  STRENGTH
  7   0.083                             50    55  STRENGTH
  11  solution for nebulization         59    84  FORM
  10  solution                          59    67  FORM
  12  nebulization                      72    84  ROUTE
  13  one (1)                           90    97  FREQUENCY
  15  neb inhalation                    98    112 ROUTE
  14  neb                               98    101 ROUTE
  16  inhalation                       102    112 ROUTE
  21  q4h (every 4 hours) as needed    113    142 FREQUENCY
  20  q4h (every 4 hours)              113    132 FREQUENCY
  17  q4h                              113    116 FREQUENCY
  19  every 4 hours                    118    131 FREQUENCY
  18  every                            118    123 FREQUENCY
  22  as needed                        133    142 FREQUENCY
  23  dyspnea                          147    154 REASON
  </code></pre>
  
  <p>Which puts the relevant row the first, however, I still need to filter out irrelevant rows....</p>