title: How to know the data type of a column in sparksql?
body: <p>I have this:</p>
  <pre class="lang-py prettyprint-override"><code>from pyspark.sql import functions as F
  df = dfc.select(list(dfc.keys())[0]).toDF()
  columns = df.columns
  df1 = df.withColumn(
          &quot;payload&quot;,
          F.to_json(F.struct([F.when(F.col(x) != &quot;  &quot;,df[x])
                               .otherwise(None).alias(x) for x in columns])))
  </code></pre>
  <p>And which columns are object and which are array.</p>