title: Apache Spark TFIDF using Python
body: <p>The Spark documentation states to use <code>HashingTF</code> feature, but I'm unsure what the transform function expects as input. 
  <a href="http://spark.apache.org/docs/latest/mllib-feature-extraction.html#tf-idf" rel="nofollow">http://spark.apache.org/docs/latest/mllib-feature-extraction.html#tf-idf</a></p>
  
  <p>I tried running the tutorial code: </p>
  
  <pre><code>from pyspark import SparkContext
  from pyspark.mllib.feature import HashingTF
  
  sc = SparkContext()
  
  # Load documents (one per line).
  documents = sc.textFile("...").map(lambda line: line.split(" "))
  
  hashingTF = HashingTF()
  tf = hashingTF.transform(documents)
  </code></pre>
  
  <p>but I get the following error:</p>
  
  <pre><code>Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
    File "/Users/salloumm/spark-1.6.0-bin-hadoop2.6/python/pyspark/ml/pipeline.py", line 114, in transform
      return self._transform(dataset)
    File "/Users/salloumm/spark-1.6.0-bin-hadoop2.6/python/pyspark/ml/wrapper.py", line 148, in _transform
      return DataFrame(self._java_obj.transform(dataset._jdf), dataset.sql_ctx)
  AttributeError: 'list' object has no attribute '_jdf'
  </code></pre>