title: Update structured values of a map type column in Pyspark
body: <p>I am trying to modify a Dataframe column of type Map the values of which are in turn of type Struct:</p>
  <pre><code>root
   |-- name: string (nullable = true)
   |-- age: long (nullable = true)
   |-- MapName: map (nullable = true)
   |    |-- key: string
   |    |-- value: struct (valueContainsNull = true)
   |    |    |-- field1: double (nullable = true)
   |    |    |-- field2: double (nullable = true)
  </code></pre>
  <p>I created the DataFrame in a way MapName column is initialised to None as I will need to fill infos in later stages. The code I used is as follows:</p>
  <pre><code>from pyspark.sql import SparkSession, DataFrame
  from pyspark.sql.functions import col, udf, lit
  from pyspark.sql.types import StructType, StringType, LongType, DoubleType, MapType
  
  spark = SparkSession.builder.appName(&quot;testMapType&quot;).getOrCreate()
  
  data = [('Alice', 1)]
  df = spark.createDataFrame(data, ['name','age'])
  
  def value_struct():
      schema = StructType([
          StructField('field1', DoubleType(), True),
          StructField('field2', DoubleType(), True)])
  
      return schema
  
  myschema = StructType([
          StructField('name', StringType(), True),
          StructField('age', LongType(), True),
          StructField('MapName', MapType(StringType(), value_struct()), True)])
  
  df = df.withColumn('MapName',lit(None).cast(MapType(StringType(), value_struct())))
  df = spark.createDataFrame(df.rdd, schema=myschema)
  df.printSchema()
  </code></pre>
  <p>Now when I try to update the MapName column using a <strong>udf</strong> like this:</p>
  <pre><code>my_update_udf = udf(lambda x: {**x, **{'a_map_key':{'field1':3.2,'field2':2.6}}}, MapType(StringType(), value_struct()))
  df2 = df.withColumn('MapName', my_update_udf(col('MapName')))
  df2.collect()
  </code></pre>
  <p>I get the following error:</p>
  <pre><code>TypeError: 'NoneType' object is not a mapping
  </code></pre>
  <p>I cannot find any suggestion from past inquires, any help here?</p>