title: Is it possible to access estimator attributes in spark.ml pipelines?
body: <p>I have a spark.ml pipeline in Spark 1.5.1 which consists of a series of transformers followed by a k-means estimator. I want to be able to access the <a href="http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.ml.clustering.KMeansModel" rel="noreferrer">KMeansModel</a>.clusterCenters after fitting the pipeline, but can't figure out how. Is there a spark.ml equivalent of sklearn's pipeline.named_steps feature? </p>
  
  <p>I found <a href="https://stackoverflow.com/a/30478509/5376571">this answer</a> which gives two options. The first works if I take the k-means model out of my pipeline and fit it separately, but that kinda defeats the purpose of a pipeline. The second option doesn't work - I get <code>error: value getModel is not a member of org.apache.spark.ml.PipelineModel</code>.</p>
  
  <p>EDIT: Example pipeline:</p>
  
  <pre><code>import org.apache.spark.ml.feature.{HashingTF, IDF, Tokenizer}
  import org.apache.spark.ml.clustering.{KMeans, KMeansModel}
  import org.apache.spark.ml.Pipeline
  
  // create example dataframe
  val sentenceData = sqlContext.createDataFrame(Seq(
    ("Hi I heard about Spark"),
    ("I wish Java could use case classes"),
    ("K-means models are neat")
    )).toDF("sentence")
  
  // initialize pipeline stages
  val tokenizer = new Tokenizer().setInputCol("sentence").setOutputCol("words")
  val hashingTF = new HashingTF().setInputCol("words").setOutputCol("features").setNumFeatures(20)
  val kmeans = new KMeans()
  val pipeline = new Pipeline().setStages(Array(tokenizer, hashingTF, kmeans))
  
  // fit the pipeline
  val fitKmeans = pipeline.fit(sentenceData)
  </code></pre>
  
  <p>So now <code>fitKmeans</code> is of type <code>org.apache.spark.ml.PipelineModel</code>. My question is, how do I access the cluster centers calculated by the k-means model contained within this pipeline? As noted above, when <strong>not</strong> contained in a pipeline, this can be done with <code>fitKmeans.clusterCenters</code>.</p>