title: "StructField" object has no attribute - "_get_object_id"
body: <p>I was trying to evaluate a random forest model by computing Precision/Recall (PR) and Receiver Operating Characteristic (ROC) values using <code>BinaryClassificationMetrics</code> from <code>pyspark.mllib.evaluation</code>, but I keep getting a <code>'StructField' object has no attribute '_get_object_id'</code>
  error.   I am using Jupyter notebook on a cluster on Amazon EMR, with optional packages Livy, TensorFlow and Spark (Sparkmagic, Pyspark); everything else is just the default.  </p>
  
  <p>The following code reproduces the error for me: </p>
  
  <pre><code>from pyspark import SparkContext   
  from pyspark.sql import SparkSession   
  import pyspark
  from pyspark.mllib.evaluation import BinaryClassificationMetrics
  
  a = sc.parallelize([
      (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
      (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
      (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
      (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
      (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
  ])  
  
  metrics = BinaryClassificationMetrics(a)
  </code></pre>
  
  <p>This is the error I get:  </p>
  
  <pre><code>'StructField' object has no attribute '_get_object_id'  
  Traceback (most recent call last):    
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/mllib/evaluation.py", line 52, in __init__ StructField("label", DoubleType(), nullable=False)]))    
  File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/context.py", line 302, in createDataFramereturn self.sparkSession.createDataFrame(data, schema, samplingRatio, verifySchema)  
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1248, in __call__args_command, temp_args = self._build_args(*args)    
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1212, in _build_args (new_args, temp_args) = self._get_args(args)   
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1199, in _get_args temp_arg converter.convert(arg,self.gateway_client)
  File "/usr/lib/spark/python/lib/py4j-0.1 src.zip/py4j/java_collections.py", line 501, in convert java_list.add(element)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1248, in __call__args_command, temp_args = self._build_args(*args)  
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1218, in _build_args [get_command_part(arg, self.pool) for arg in new_args])   
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 298, in get_command_part command_part = REFERENCE_TYPE + parameter._get_object_id()  
  AttributeError: 'StructField' object has no attribute '_get_object_id'
  </code></pre>