title: How can you parse a string that is json from an existing temp table using PySpark?
body: <p>I have an existing Spark dataframe that has columns as such:</p>
  
  <pre><code>--------------------
  pid | response
  --------------------
   12 | {"status":"200"}
  </code></pre>
  
  <p>response is a string column.
  Is there a way to cast it into JSON and extract specific fields? Can lateral view be used as it is in Hive? I looked up some examples on line that used explode and later view but it doesn't seem to work with Spark 2.1.1</p>