title: Operating on Pandas groups
body: <p>I am attempting to perform multiple operations on a large dataframe (~3 million rows).<br>
  Using a small test-set representative of my data, I've come up with a solution. <br>However the script runs extremely slowly when using the large dataset as input.<br></p>
  
  <p>Here is the main loop of the application:<br></p>
  
  <pre><code>def run():
      df = pd.DataFrame(columns=['CARD_NO','CUSTOMER_ID','MODIFIED_DATE','STATUS','LOYALTY_CARD_ENROLLED'])
  
      foo = input.groupby('CARD_NO', as_index=False, sort=False)
  
      for name, group in foo:
          if len(group) == 1:
              df = df.append(group)
          else:
              dates = group['MODIFIED_DATE'].values
              if all_same(dates):
                  df = df.append(group[group.STATUS == '1'])
              else:
                  df = df.append(group[group.MODIFIED_DATE == most_recent(dates)])
  
      path = ''
      df.to_csv(path, sep=',', index=False)
  </code></pre>
  
  <p>The logic is as follows:<br>
  For each CARD_NO<br>
  - if there is only 1 CARD_NO, add row to new dataframe<br>
  - if there are > 1 of the same CARD_NO, check MODIFIED_DATE,<br>
  - if MODIFIED_DATEs are different, take the row with most recent date<br>
  - if all MODIFIED_DATES are equal, take whichever row has STATUS = 1<br></p>
  
  <p>The slow-down occurs at each iteration around, </p>
  
  <pre><code>input.groupby('CARD_NO', as_index=False, sort=False)
  </code></pre>
  
  <p>I am currently trying to parallelize the loop by splitting the groups returned by the above statement, but I'm not sure if this is the correct approach... <br></p>
  
  <p>Am I overlooking a core functionality of Pandas?<br>
  Is there a better, more Pandas-esque way of solving this problem?<br></p>
  
  <p>Any help is greatly appreciated.<br>
  Thank you.</p>