title: Extract Json data from a spark dataframe
body: <pre><code>+------------------------------------------------------------------+
  | message                                                          |
  +------------------------------------------------------------------+
  |{"name":"east-desktop","viewers":447,"emptyCount":0,"version":0.3}|
  |{"name":"west-desktop","viewers":111,"emptyCount":0,"version":0.6}|
  |{"name":"west-desktop","viewers":115,"emptyCount":0,"version":0.1}|
  +------------------------------------------------------------------+
  
  message:string
  </code></pre>
  
  <p>I have a dataframe which contains json data within one column, I would like to extract the data in to either separate columns or as json file.</p>
  
  <p>I am working within a Databricks notebook using pyspark.</p>
  
  <p>Dataframe</p>
  
  <pre><code>+---------------------------------------------+
  | name        | viewers| emptyCount | version |
  +---------------------------------------------+
  |east-desktop | 447    | 0          | 0.3     |
  |west-desktop | 111    | 0          | 0.6     |
  |west-desktop | 115    | 0          | 0.1     |
  +---------------------------------------------+
  </code></pre>
  
  <p>OR Json</p>
  
  <pre><code>{
    "name": "east-desktop",
    "viewers":  447,
    "emptyCount": 0,
    "version": 0.3,
  }
  </code></pre>