title: Deserialize Kafka Thrift message with PySpark Streaming to JSON
body: <p>I am using databricks environment and reading a Kafka input.</p>
  <p>The messages consumed from Kafka are in Thrift Binary and am having issues deserialize it to JSON.</p>
  <p>I am writing an UDF to do this, but am unable to figure out how to convert thrift -&gt; JSON?</p>
  <p>I have tried</p>
  <pre><code>import thriftpy2.protocol.json as proto
      
  def decoder(thrift_data):
    return proto.struct_to_json(thrift_data)
  
  which throws an error PythonException: 'AttributeError: 'bytearray' object has no attribute 'thrift_spec''
  
  </code></pre>
  <p>and also tried</p>
  <pre><code>def decoder(thrift_data):
    return serialize(thrift_data, protocol_factory=TSimpleJSONProtocolFactory())
  
  which throws an error PythonException: 'AttributeError: 'bytearray' object has no attribute 'write'',
  </code></pre>