title: Optimizing pandas groupby on many small groups
body: <p>I have a pandas DataFrame with many small groups:</p>
  
  <pre><code>In [84]: n=10000
  
  In [85]: df=pd.DataFrame({'group':sorted(range(n)*4),'val':np.random.randint(6,size=4*n)}).sort(['group','val']).reset_index(drop=True)
  
  In [86]: df.head(9)
  Out[86]: 
     group  val
  0      0    0
  1      0    0
  2      0    1
  3      0    2
  4      1    1
  5      1    2
  6      1    2
  7      1    4
  8      2    0
  </code></pre>
  
  <p>I want to do something special for groups where val==1 appears but not val==0. E.g. replace the 1 in the group by 99 only if the val==0 is in that group.</p>
  
  <p>But for DataFrames of this size it is quite slow:</p>
  
  <pre><code>In [87]: def f(s):
     ....: if (0 not in s) and (1 in s): s[s==1]=99
     ....: return s
     ....: 
  
  In [88]: %timeit df.groupby('group')['val'].transform(f)
  1 loops, best of 3: 11.2 s per loop
  </code></pre>
  
  <p>Looping through the data frame is much uglier but much faster:</p>
  
  <pre><code>In [89]: %paste
  
  def g(df):
      df.sort(['group','val'],inplace=True)
      last_g=-1
      for i in xrange(len(df)):
          if df.group.iloc[i]!=last_g:
              has_zero=False
          if df.val.iloc[i]==0:
              has_zero=True
          elif has_zero and df.val.iloc[i]==1:
              df.val.iloc[i]=99
      return df
  ## -- End pasted text --
  
  In [90]: %timeit g(df)
  1 loops, best of 3: 2.53 s per loop
  </code></pre>
  
  <p>But I would like to optimizing it further if possible.</p>
  
  <p>Any idea of how to do so?</p>
  
  <p>Thanks</p>
  
  <hr>
  
  <p>Based on Jeff's answer, I got a solution that is very fast. I'm putting it here if others find it useful:</p>
  
  <pre><code>In [122]: def do_fast(df):
     .....: has_zero_mask=df.group.isin(df[df.val==0].group.unique())
     .....: df.val[(df.val==1) &amp; has_zero_mask]=99
     .....: return df
     .....: 
  
  In [123]: %timeit do_fast(df)
  100 loops, best of 3: 11.2 ms per loop
  </code></pre>