title: How to load CSVs with timestamps in custom format?
body: <p>I have a timestamp field in a csv file that I load to a dataframe using spark csv library. The same piece of code works on my local machine with Spark 2.0 version but  throws an error on Azure Hortonworks HDP 3.5 and 3.6.</p>
  
  <p>I have checked and Azure HDInsight 3.5 is also using the same Spark version so I don't think it's a problem with Spark version.</p>
  
  <pre><code>import org.apache.spark.sql.types._
  val sourceFile = "C:\\2017\\datetest"
  val sourceSchemaStruct = new StructType()
    .add("EventDate",DataTypes.TimestampType)
    .add("Name",DataTypes.StringType)
  val df = spark.read
    .format("com.databricks.spark.csv")
    .option("header","true")
    .option("delimiter","|")
    .option("mode","FAILFAST")
    .option("inferSchema","false")
    .option("dateFormat","yyyy/MM/dd HH:mm:ss.SSS")
    .schema(sourceSchemaStruct)
    .load(sourceFile)
  </code></pre>
  
  <p>The whole exception is as follows:</p>
  
  <pre><code>Caused by: java.lang.IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
    at java.sql.Timestamp.valueOf(Timestamp.java:237)
    at org.apache.spark.sql.catalyst.util.DateTimeUtils$.stringToTime(DateTimeUtils.scala:179)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9$$anonfun$apply$13$$anonfun$apply$2.apply$mcJ$sp(UnivocityParser.scala:142)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9$$anonfun$apply$13$$anonfun$apply$2.apply(UnivocityParser.scala:142)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9$$anonfun$apply$13$$anonfun$apply$2.apply(UnivocityParser.scala:142)
    at scala.util.Try.getOrElse(Try.scala:79)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9$$anonfun$apply$13.apply(UnivocityParser.scala:139)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9$$anonfun$apply$13.apply(UnivocityParser.scala:135)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser.org$apache$spark$sql$execution$datasources$csv$UnivocityParser$$nullSafeDatum(UnivocityParser.scala:179)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9.apply(UnivocityParser.scala:135)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$makeConverter$9.apply(UnivocityParser.scala:134)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser.org$apache$spark$sql$execution$datasources$csv$UnivocityParser$$convert(UnivocityParser.scala:215)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser.parse(UnivocityParser.scala:187)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$5.apply(UnivocityParser.scala:304)
    at org.apache.spark.sql.execution.datasources.csv.UnivocityParser$$anonfun$5.apply(UnivocityParser.scala:304)
    at org.apache.spark.sql.execution.datasources.FailureSafeParser.parse(FailureSafeParser.scala:61)
    ... 27 more
  </code></pre>
  
  <p>The csv file has only one row as follows:</p>
  
  <pre><code>"EventDate"|"Name"
  "2016/12/19 00:43:27.583"|"adam"
  </code></pre>