title: Find the relative weeks between two dates
body: <p>I have a spark dataframe with 2 columns which represent dates (date1 and date2). I would like to find the relative number of weeks between the two dates (+ 1 week). In order to do that I find the week of each date and subtract the two. date2 is always an earlier date. The largest difference between the dates is at most 1 year so if date2 is from a previous year I need to add 52 to the solution. </p>
  
  <p>I figured out how to do it but my solution seems cumbersome. Here is my solution:</p>
  
  <pre><code>from pyspark.sql.functions import date_format, udf
  from pyspark.sql.types import IntegerType
  
  def fixWeeks(w):
    if w &lt; 0:
      w = w + 52
    return w
  
  week_filter = udf(fixWeeks,IntegerType())
  df_week = df.withColumn("TmpWeek", date_format("date1",'w').cast('int')-date_format("date2",'w').cast('int')+1)
  df_week = df_week.withColumn('week',week_filter(df_week.TmpWeek)).drop('TmpWeek')
  </code></pre>
  
  <p>Is there a better way to achieve the same solution? How can I do the same thing in a single function? </p>