title: 'DataFrame' object has no attribute 'withColumn'
body: <p>I am trying to compare two pandas dataframes but I get an error as  'DataFrame' object has no attribute 'withColumn'. What could be the issue?</p>
  
  <pre><code>import pandas as pd
  import pyspark.sql.functions as F
  
  pd_df=pd.DataFrame(df.dtypes,columns=['column','data_type'])
  pd_df1=pd.DataFrame(df1.dtypes,columns=['column','data_type'])
  
  pd.merge(pd_df,pd_df1, on='column', how='outer'
      ).withColumn(
      "result",
      F.when(F.col("data_type_x") == 'NaN','new attribute'.otherwise('old attribute')))
      .select(
      "column",
      "data_type_x",
      "data_type_y",
      "result"
      )
  </code></pre>
  
  <p>df and df1 are some data frames</p>