title: Pyspark for converting a column String to Date
body: <p>Im trying to convert a &quot;dob&quot; column from string to date datatype to do some basic operations in pyspark. My Input is</p>
  <pre><code>  long_name                         age dob         wage_eur
  Cristiano Ronaldo dos Santos Aveiro 32  05-02-1985  565000
  Lionel AndrÃ©s Messi Cuccittini     30  24-06-1987  565000
  </code></pre>
  <p>I created a customized schema to change the dob column from string to date type and used to query the dataframe as follows</p>
  <pre><code>spark.sql(&quot;set spark.sql.legacy.timeParserPolicy=LEGACY&quot;)
  from pyspark.sql.types import IntegerType , DateType , StringType , TimestampType, StructType , StructField
  peopleschema = StructType([StructField(&quot;long_name&quot;,StringType(),True),
                             StructField(&quot;age&quot;,IntegerType(),True),
                             **StructField(&quot;dob&quot;,DateType(),True),**
                             StructField(&quot;wage_eur&quot;,IntegerType(),True)                          
                            ])
  
  file_location = &quot;/FileStore/tables/Fifa_data_Dateconvertion-1.csv&quot;
  file_type = &quot;csv&quot;
  
  infer_schema = &quot;false&quot;
  first_row_is_header = &quot;true&quot;
  delimiter = &quot;,&quot;
  
  df = spark.read.format(file_type) \
    .option(&quot;inferSchema&quot;, infer_schema) \
    .option(&quot;header&quot;, first_row_is_header) \
    .option(&quot;sep&quot;, delimiter) \
    **.schema(peopleschema) \**
    .load(file_location)
  
  display(df)
  
  </code></pre>
  <p>The schema got converted for dob column but the values are coming as follows</p>
  <pre><code>long_name:string
  age:integer
  dob:date
  wage_eur:integer
  
  
  long_name                           age dob         wage_eur
  Cristiano Ronaldo dos Santos Aveiro 32  0010-07-09  565000
  Lionel AndrÃ©s Messi Cuccittini     30  0029-11-08  565000
  </code></pre>
  <p>So when i retrieve the year alone by querying the data frame im getting wrong values</p>
  <pre><code>from pyspark.sql.functions import col
  import datetime
  from pyspark.sql.functions import year
  from pyspark.sql.functions import to_date
  
  df1 = df.withColumn('birth_year',year(df.dob))
  
  df1.show()
  
  </code></pre>
  <p>Im getting the result as</p>
  <pre><code>           long_name|age|       dob|wage_eur|birth_year|
  +--------------------+---+----------+--------+----------+
  |Cristiano Ronaldo...| 32|0010-07-09|  565000|        10|
  |Lionel Andrés Mes...| 30|0029-11-08|  565000|        29|
  
  </code></pre>
  <p>Can someone guide me ?</p>
  <p>Thanks,
  Aarush</p>