title: Pyspark windows on last 30 days on subset of data
body: <p>I have a working Pyspark Windowing function (Spark 2.0) that takes the last 30 days (86400*30) seconds and counts the number of times each action in column 'a' happens per ID. The dataset that I am applying this function to has multiple records for every day between '2018-01-01' and '2018-04-01'. Because this is a 30 day look back, I don't want to apply this function to data that doesn't have a full 30 days to look back on. For convenience, I want to start my counts on Feb 1st. I can' filter out January, because it is needed for Februrary's counts. I know I can just throw a filter on the new dataframe and filter out the data before for February, but is there a way to do it without that extra step? It'd be nice to not have to preform the calculations which could save time.</p>
  
  <p>Here's the code:</p>
  
  <pre><code>from pyspark.sql import Window
  from pyspark.sql import functions as F
  
  windowsess = Window.partitionBy("id",'a').orderBy('ts').rangeBetween(-86400*30, Window.currentRow)
  
  df4 = df3.withColumn("2h4_ct",F.count(df.a).over(windowsess))
  </code></pre>
  
  <p>Mockup of current dataset. I didn't want to convert the col ts, by hand so I wrote in a substitute for it.</p>
  
  <pre><code>id,a,timestamp,ts
  1,soccer,2018-01-01 10:41:00, &lt;unix_timestamp&gt; 
  1,soccer,2018-01-13 10:40:00, &lt;unix_timestamp&gt; 
  1,soccer,2018-01-23 10:39:00, &lt;unix_timestamp&gt; 
  1,soccer,2018-02-01 10:38:00, &lt;unix_timestamp&gt; 
  1,soccer,2018-02-03 10:37:00, &lt;unix_timestamp&gt; 
  1,leagueoflegends,2018-02-04 10:36:00, &lt;unix_timestamp&gt;
  </code></pre>
  
  <p>With my made up sample data. I want to return the following rows</p>
  
  <pre><code>1,soccer,2018-02-01 10:38:00, &lt;unix_timestamp&gt;,4
  1,soccer,2018-02-03 10:37:00, &lt;unix_timestamp&gt;,5
  1,leagueoflegends,2018-02-04 10:36:00, &lt;unix_timestamp&gt;,1
  </code></pre>
  
  <p>instead I get this:</p>
  
  <pre><code>1,soccer,2018-01-01 10:41:00, &lt;unix_timestamp&gt;,1
  1,soccer,2018-01-13 10:40:00, &lt;unix_timestamp&gt;,2
  1,soccer,2018-01-23 10:39:00, &lt;unix_timestamp&gt;,3
  1,soccer,2018-02-01 10:38:00, &lt;unix_timestamp&gt;,4
  1,soccer,2018-02-03 10:37:00, &lt;unix_timestamp&gt;,5
  1,leagueoflegends,2018-02-04 10:36:00, &lt;unix_timestamp&gt;,1
  </code></pre>