title: How to convert Timestamp column to milliseconds Long column in Spark SQL
body: <p>What is the shortest and the most efficient way in Spark SQL to transform <code>Timestamp</code> column to a milliseconds timestamp <code>Long</code> column?</p>
  <p>Here is an example of a transformation from timestamp to milliseconds</p>
  <pre><code>scala&gt; val ts = spark.sql(&quot;SELECT now() as ts&quot;)
  ts: org.apache.spark.sql.DataFrame = [ts: timestamp]
  
  scala&gt; ts.show(false)
  +-----------------------+                                                       
  |ts                     |
  +-----------------------+
  |2019-06-18 12:32:02.41 |
  +-----------------------+
  
  scala&gt; val tss = ts.selectExpr(
   |   &quot;ts&quot;,
   |   &quot;BIGINT(ts) as seconds_ts&quot;,
   |   &quot;BIGINT(ts) * 1000 + BIGINT(date_format(ts, 'SSS')) as millis_ts&quot;
   | )
  tss: org.apache.spark.sql.DataFrame = [ts: timestamp, seconds_ts: bigint ... 1 more field]
  
  scala&gt; tss.show(false)
  +----------------------+----------+-------------+                               
  |ts                    |seconds_ts|millis_ts    |
  +----------------------+----------+-------------+
  |2019-06-18 12:32:02.41|1560861122|1560861122410|
  +----------------------+----------+-------------+
  </code></pre>
  <p>As you can see, the most straightforward method to get milliseconds from timestamp doesn't work - cast to long returns seconds, however milliseconds information in timestamp is preserved.</p>
  <p>The only way I found to to extract milliseconds information is by using <code>date_format</code> function , which is nothing like as simple as I would expect.</p>
  <p>Does anybody know the way to get milliseconds UNIX time out of <code>Timestamp</code> column simpler than that?</p>