title: From DataFrame to RDD[LabeledPoint]
body: <p>I am trying to implement a document classifier using Apache Spark MLlib and I am having some problems representing the data. My code is the following:</p>
  
  <pre><code>import org.apache.spark.sql.{Row, SQLContext}
  import org.apache.spark.sql.types.{StringType, StructField, StructType}
  import org.apache.spark.ml.feature.Tokenizer
  import org.apache.spark.ml.feature.HashingTF
  import org.apache.spark.ml.feature.IDF
  
  val sql = new SQLContext(sc)
  
  // Load raw data from a TSV file
  val raw = sc.textFile("data.tsv").map(_.split("\t").toSeq)
  
  // Convert the RDD to a dataframe
  val schema = StructType(List(StructField("class", StringType), StructField("content", StringType)))
  val dataframe = sql.createDataFrame(raw.map(row =&gt; Row(row(0), row(1))), schema)
  
  // Tokenize
  val tokenizer = new Tokenizer().setInputCol("content").setOutputCol("tokens")
  val tokenized = tokenizer.transform(dataframe)
  
  // TF-IDF
  val htf = new HashingTF().setInputCol("tokens").setOutputCol("rawFeatures").setNumFeatures(500)
  val tf = htf.transform(tokenized)
  tf.cache
  val idf = new IDF().setInputCol("rawFeatures").setOutputCol("features")
  val idfModel = idf.fit(tf)
  val tfidf = idfModel.transform(tf)
  
  // Create labeled points
  val labeled = tfidf.map(row =&gt; LabeledPoint(row.getDouble(0), row.get(4)))
  </code></pre>
  
  <p>I need to use dataframes to generate the tokens and create the TF-IDF features. The problem appears when I try to convert this dataframe to a RDD[LabeledPoint]. I map the dataframe rows, but the get method of Row return an Any type, not the type defined on the dataframe schema (Vector). Therefore, I cannot construct the RDD I need to train a ML model.</p>
  
  <p>What is the best option to get a RDD[LabeledPoint] after calculating a TF-IDF?</p>