title: Pyspark Specify Default Value for Column
body: <p>I'm using Spark 1.6.1 and Python 2.7</p>
  
  <p>I'm trying to figure out how I can specify a default value for a newly added column to a dataframe. Take this scenario where I have a dataframe named df1 that contains</p>
  
  <pre><code>+-------+----+
  |user_id| age|
  +-------+----+
  |  10000|  45|
  |  10013|  40|
  |  10021|Null|
  |  10025|  50|
  |  10051|  31|
  +-------+----+
  </code></pre>
  
  <p>Now I want to add a new column called age2 that just has a simple function age + 1</p>
  
  <pre><code>&gt;&gt;&gt; df1 = df1.withColumn("age2", df1["age"]+1)
  
  +-------+----+----+
  |user_id| age|age2|
  +-------+----+----+
  |  10000|  45|  46|
  |  10013|  40|  41|
  |  10021|Null|Null|
  |  10025|  50|  51|
  |  10051|  31|  32|
  +-------+----+----+
  </code></pre>
  
  <p>Is there a way I can specify a default value for age2 so that instead of returning a null for age2 when there is a null in age, I can return something like 1 so that I get</p>
  
  <pre><code>+-------+----+----+
  |user_id| age|age2|
  +-------+----+----+
  |  10000|  45|  46|
  |  10013|  40|  41|
  |  10021|Null|   1|
  |  10025|  50|  51|
  |  10051|  31|  32|
  +-------+----+----+
  </code></pre>
  
  <p>I know that I can use a UDF to do this, but I want to know if there is a built in way to do it instead.</p>