title: Identifying consecutive NaNs with Pandas
body: <p>I am reading in a bunch of CSV files (measurement data for water levels over time) to do various analysis and visualizations on them.</p>
  <p>Due to various reasons beyond my control, these time series often have missing data, so I do two things:</p>
  <p>I count them in total with</p>
  <pre><code>Rlength = len(RainD)   # Counts everything, including NaN
  Rcount = RainD.count() # Counts only valid numbers
  NaN_Number = Rlength - Rcount
  </code></pre>
  <p>and discard the dataset if I have more missing data than a certain threshold:</p>
  <pre><code>Percent_Data = Rlength/100
  Five_Percent = Percent_Data*5
  if NaN_Number &gt; Five_Percent:
      ...
  </code></pre>
  <p>If the number of NaN is sufficiently small, I would like to fill the gaps with</p>
  <pre><code>RainD.level = RainD.level.fillna(method='pad', limit=2)
  </code></pre>
  <p>And now for the issue: It's monthly data, so if I have more than two consecutive NaNs, I also want to discard the data, since that would mean that I &quot;guess&quot; a whole season, or even more.</p>
  <p>The <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html#pandas.DataFrame.fillna" rel="nofollow noreferrer">documentation for <code>fillna</code></a> doesn't really mention what happens when there is more consecutive NaNs than my specified <code>limit=2</code>, but when I look at <code>RainD.describe()</code> before and after <code>...fillna...</code> and compare it with the base CSV, it's clear that it fills the first two NaNs, and then leaves the rest as it is, instead of erroring out.</p>
  <p>So, long story short:</p>
  <p><strong>How do I identify a number of consecutive NaNs with Pandas, without some complicated and time consuming non-Pandas loop?</strong></p>