title: Input type must be string type but got ArrayType(StringType,true) error in Spark using Scala
body: <p>I am new to Spark and am using Scala to create a basic classifier. I am reading from a textfile as a dataset and splitting it into training and test data sets. Then I'm trying to tokenize the training data but it fails with </p>
  
  <pre><code>Caused by: java.lang.IllegalArgumentException: requirement failed: Input type must be string type but got ArrayType(StringType,true).
  at scala.Predef$.require(Predef.scala:224)
  at org.apache.spark.ml.feature.RegexTokenizer.validateInputType(Tokenizer.scala:149)
  at org.apache.spark.ml.UnaryTransformer.transformSchema(Transformer.scala:110)
  at org.apache.spark.ml.Pipeline$$anonfun$transformSchema$4.apply(Pipeline.scala:180)
  at org.apache.spark.ml.Pipeline$$anonfun$transformSchema$4.apply(Pipeline.scala:180)
  at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:57)
  at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:66)
  at scala.collection.mutable.ArrayOps$ofRef.foldLeft(ArrayOps.scala:186)
  at org.apache.spark.ml.Pipeline.transformSchema(Pipeline.scala:180)
  at org.apache.spark.ml.PipelineStage.transformSchema(Pipeline.scala:70)
  at org.apache.spark.ml.Pipeline.fit(Pipeline.scala:132)
  at com.classifier.classifier_app.App$.&lt;init&gt;(App.scala:91)
  at com.classifier.classifier_app.App$.&lt;clinit&gt;(App.scala)
  ... 1 more
  </code></pre>
  
  <p>error. </p>
  
  <p>The code is as below:</p>
  
  <pre><code>val input_path = "path//to//file.txt"
  
  case class Sentence(value: String)
  val sentencesDS = spark.read.textFile(input_path).as[Sentence]  
  
  val Array(trainingData, testData) = sentencesDS.randomSplit(Array(0.7, 0.3))     
  
  val tokenizer = new Tokenizer()
    .setInputCol("value")
    .setOutputCol("words")
  
  val pipeline = new Pipeline().setStages(Array(tokenizer, regexTokenizer, remover, hashingTF, ovr))
  val model = pipeline.fit(trainingData)
  </code></pre>
  
  <p>How do I solve this? Any help is appreciated.</p>
  
  <p>I have defined all the stages in the pipeline but haven't put them here in the code snippet.</p>