title: Python pandas: insert rows for missing dates, time series in groupby dataframe
body: <p>I have a dataframe <code>df</code>:</p>
  <pre><code>   Serial_no       date  Index     x    y
             1 2014-01-01      1   2.0  3.0
             1 2014-03-01      2   3.0  3.0
             1 2014-04-01      3   6.0  2.0
             2 2011-03-01      1   5.1  1.3
             2 2011-04-01      2   5.8  0.6
             2 2011-05-01      3   6.5 -0.1
             2 2011-07-01      4   3.0  5.0
             3 2019-10-01      1   7.9 -1.5
             3 2019-11-01      2   8.6 -2.2
             3 2020-01-01      3  10.0 -3.6
             3 2020-02-01      4  10.7 -4.3
             3 2020-03-01      5   4.0  3.0
  </code></pre>
  <p><strong>Notice:</strong>
  The data is grouped by <code>Serial_no</code> and the <code>date</code> is data reported monthly (first of every month).
  The <code>Index</code> column is set so each consecutive reported date is a consecutive number in the series.
  The number of reported dates in each group <code>Serial_no</code> are different.
  The interval of reported dates <code>date</code> are different for each group <code>Serial_no</code> (they don't start or end on the same date for each group).</p>
  <p><strong>The problem:</strong>
  There is no reported data for some dates <code>date</code> in the time series. Notice some dates are missing in each <code>Serial_no</code> group.
  I want to add a row in each group for those missing dates <code>date</code> and have the data reported in <code>x</code> and <code>y</code> columns as 'NaN'.</p>
  <p><strong>Example of the dataframe I need:</strong></p>
  <pre><code>   Serial_no       date  Index       x       y
             1 2014-01-01      1     2.0     3.0
             1 2014-02-01      2     NaN     NaN
             1 2014-03-01      3     3.0     3.0
             1 2014-04-01      4     6.0     2.0
             2 2011-03-01      1     5.1     1.3
             2 2011-04-01      2     5.8     0.6
             2 2011-05-01      3     6.5    -0.1
             2 2011-06-01      4     NaN     NaN
             2 2011-07-01      5     3.0     5.0
             3 2019-10-01      1     7.9    -1.5
             3 2019-11-01      2     8.6    -2.2
             3 2019-12-01      3     NaN     NaN
             3 2020-01-01      4    10.0    -3.6
             3 2020-02-01      5    10.7    -4.3
             3 2020-03-01      6     4.0     3.0
  </code></pre>
  <p><strong>I know how to replace the blank cells with <code>NaN</code> once the rows with missing dates are inserted, using the following code:</strong></p>
  <pre><code>import pandas as pd
  import numpy as np
  
  df['x'].replace('', np.nan, inplace=True)
  df['y'].replace('', np.nan, inplace=True)
  </code></pre>
  <p><strong>I also know how to reset the index once the rows with missing dates are inserted, using the following code:</strong></p>
  <pre><code>df[&quot;Index&quot;] = df.groupby(&quot;Serial_no&quot;,).cumcount('date')
  </code></pre>
  <p>However, I'm unsure how to locate the the missing dates in each group and insert the row for those (monthly reported) dates. Any help is appreciated.</p>