title: How to convert a dataframe to dataset in Apache Spark in Scala?
body: <p>I need to convert my dataframe to a dataset and I used the following code:</p>
  
  <pre><code>    val final_df = Dataframe.withColumn(
        "features",
        toVec4(
          // casting into Timestamp to parse the string, and then into Int
          $"time_stamp_0".cast(TimestampType).cast(IntegerType),
          $"count",
          $"sender_ip_1",
          $"receiver_ip_2"
        )
      ).withColumn("label", (Dataframe("count"))).select("features", "label")
  
      final_df.show()
  
      val trainingTest = final_df.randomSplit(Array(0.3, 0.7))
      val TrainingDF = trainingTest(0)
      val TestingDF=trainingTest(1)
      TrainingDF.show()
      TestingDF.show()
  
      ///lets create our liner regression
      val lir= new LinearRegression()
      .setRegParam(0.3)
      .setElasticNetParam(0.8)
      .setMaxIter(100)
      .setTol(1E-6)
  
      case class df_ds(features:Vector, label:Integer)
      org.apache.spark.sql.catalyst.encoders.OuterScopes.addOuterScope(this)
  
      val Training_ds = TrainingDF.as[df_ds]
  </code></pre>
  
  <p>My problem is that, <strong>I got the following error:</strong></p>
  
  <pre><code>Error:(96, 36) Unable to find encoder for type stored in a Dataset.  Primitive types (Int, String, etc) and Product types (case classes) are supported by importing spark.implicits._  Support for serializing other types will be added in future releases.
      val Training_ds = TrainingDF.as[df_ds]
  </code></pre>
  
  <p>It seems that the number of values in dataframe is different with the number of value in my class. However I am using <code>case class df_ds(features:Vector, label:Integer)</code> on my TrainingDF dataframe since, It has a vector of features and an integer label. Here is TrainingDF dataframe:</p>
  
  <pre><code>+--------------------+-----+
  |            features|label|
  +--------------------+-----+
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,19...|   19|
  |[1.497325796E9,10...|   10|
  +--------------------+-----+
  </code></pre>
  
  <p>Also here is my original <strong>final_df</strong> dataframe:</p>
  
  <pre><code>+------------+-----------+-------------+-----+
  |time_stamp_0|sender_ip_1|receiver_ip_2|count|
  +------------+-----------+-------------+-----+
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.2|     10.0.0.3|   19|
  |    05:49:56|   10.0.0.3|     10.0.0.2|   10|
  +------------+-----------+-------------+-----+
  </code></pre>
  
  <p>However I got the mentioned error! Can anybody help me?
  Thanks in advance. </p>