title: Difference of two TimestampTypes in pyspark
body: <p>Using pyspark, I have a data frame with two <code>TimestampType</code> columns:</p>
  
  <pre class="lang-python prettyprint-override"><code>df.schema
  ...StructField(session_start,TimestampType,true),StructField(session_end,TimestampType,true)...
  </code></pre>
  
  <p>But I can't figure out how to compute the difference:</p>
  
  <pre class="lang-python prettyprint-override"><code>df2 = df.withColumn("session_length",col("session_end")-col("session_start"))
  </code></pre>
  
  <p>gives me</p>
  
  <pre class="lang-python prettyprint-override"><code>AnalysisException: u"cannot resolve '(`session_end` - `session_start`)' due to data type mismatch: '(`session_end` - `session_start`)' requires (numeric or calendarinterval) type, not timestamp ...
  </code></pre>
  
  <p>I haven't found an alternative that works.  (There's a <code>datediff</code> function, but that returns a result in days, and I need the difference in seconds.)</p>
  
  <p>How should I write this?</p>
  
  <p><strong>EDIT:</strong> The original version of this question had a different error caused by reusing <code>col</code> as a variable in my notebook.  After redoing the <code>import</code> to get the function back, I now get the above <code>AnalysisException</code>.</p>