title: Spark UDF - Return type DataFrame
body: <p>I have created a UDF which will add a column flag in DataFrame and return new dataFrame.</p>
  
  <pre><code>def find_mismatch = udf((df: DataFrame) =&gt; {
  df.withColumn("Flag",when(df("T_RTR_NUM").isNull &amp;&amp; df("P_RTR_NUM").isNull , 
  "Present in Flex but missing Trn and Platform"))
  
  }
  )
  </code></pre>
  
  <p>I am able to create UDF but when I pass a DataFrame into this , it gets errored out.
  It works with normal function but when it comes to Spark UDF , it gets errored out.</p>
  
  <p>Also, help me in understanding what difference will it make If I use normal function instead of spark UDF.</p>
  
  <p>Please help. I have attached screenshot of code.
  <a href="https://i.stack.imgur.com/XRL5B.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XRL5B.png" alt="enter image description here"></a></p>