title: Spark UDF thread safety
body: <p>I'm using Spark to take a dataframe containing a column of dates, and create 3 new columns containing the time in days, weeks, and months between the date in the column and today.</p>
  
  <p>My concern is around the use of SimpleDateFormat, which isn't thread safe. Ordinarily without Spark this would be fine since it's a local variable, but with Spark's lazy evaluation, is sharing a single SimpleDateFormat instance over multiple UDFs likely to cause an issue?</p>
  
  <pre><code>def calcTimeDifference(...){
      val sdf = new SimpleDateFormat(dateFormat)
  
      val dayDifference = udf{(x: String) =&gt; math.abs(Days.daysBetween(new DateTime(sdf.parse(x)), presentDate).getDays)}
      output = output.withColumn("days", dayDifference(myCol))
  
      val weekDifference = udf{(x: String) =&gt; math.abs(Weeks.weeksBetween(new DateTime(sdf.parse(x)), presentDate).getWeeks)}
      output = output.withColumn("weeks", weekDifference(myCol))
  
      val monthDifference = udf{(x: String) =&gt; math.abs(Months.monthsBetween(new DateTime(sdf.parse(x)), presentDate).getMonths)}
      output = output.withColumn("months", monthDifference(myCol))
  }
  </code></pre>