title: Why I get null results from date_format() PySpark function?
body: <p>Suppose there is a dateframe with a column comprised of dates as strings. For that assumption, we create the following dataFrame as an example:</p>
  
  <pre class="lang-py prettyprint-override"><code># Importing sql types
  from pyspark.sql.types import StringType, IntegerType, StructType, StructField, DoubleType, FloatType, DateType
  from pyspark.sql.functions  import date_format
  import random
  import time
  
  def strTimeProp(start, end, format, prop):
      stime = time.mktime(time.strptime(start, format)) # Parse a string representing a time according to a format
      etime = time.mktime(time.strptime(end, format)) 
      ptime = stime + prop * (etime - stime) 
      return time.strftime(format, time.localtime(ptime))
  
  def randomDate(start, end, prop):
      return strTimeProp(start, end, '%m-%d-%Y', prop)
  
  # Creación de un dataframe de prueba:
  schema = StructType(
          [
       StructField("dates1", StringType(), True),
           StructField("dates2",  StringType(), True)
      ]
  )
  
  size = 32
  numCol1 = [str(randomDate("1-1-1991", "1-1-1992", random.random())) for number in range(size)]
  numCol2 = [str(randomDate("1-1-1991", "1-1-1992", random.random())) for number in range(size)]
  # Building dataFrame:
  sqlContext = SQLContext(sc)
  df = sqlContext.createDataFrame(list(zip(numCol1, numCol2)),schema=schema)
  df.show(5)
  </code></pre>
  
  <p>In the code above, a random date column is generated, here is an example:</p>
  
  <pre><code>+----------+----------+
  |    dates1|    dates2|
  +----------+----------+
  |12-21-1991|05-30-1991|
  |05-28-1991|01-23-1991|
  |03-01-1991|08-05-1991|
  |07-15-1991|05-13-1991|
  |07-21-1991|11-10-1991|
  +----------+----------+
  </code></pre>
  
  <p>What I am trying to do is to change date format with the following code (from pySpark documentation):</p>
  
  <pre class="lang-py prettyprint-override"><code># Changing date formats:
  df.select(date_format('dates1', 'MM-dd-yyy').alias('newFormat')).show(5)
  </code></pre>
  
  <p>But I get this bad result:</p>
  
  <pre><code>+---------+
  |newFormat|
  +---------+
  |     null|
  |     null|
  |     null|
  |     null|
  |     null|
  +---------+
  </code></pre>
  
  <p>I suppose there is a problem relate with the string dataType but at same time, I don't understand why this code bellow works and the code above don't. </p>
  
  <pre class="lang-py prettyprint-override"><code>fechas = ['1000-01-01', '1000-01-15']
  df = sqlContext.createDataFrame(list(zip(fechas, fechas)), ['dates', 'd'])
  df.show()
  
  # Changing date formats:
  df.select(date_format('dates', 'MM-dd-yyy').alias('newFormat')).show()
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>+----------+----------+
  |     dates|         d|
  +----------+----------+
  |1000-01-01|1000-01-01|
  |1000-01-15|1000-01-15|
  +----------+----------+
  
  +----------+
  | newFormat|
  +----------+
  |01-01-1000|
  |01-15-1000|
  +----------+
  </code></pre>
  
  <p>This last results is what I want. </p>