title: How can I create multiple Datasets with different class types from one general Dataset?
body: <p>I have a <code>Dataset[General]</code> which I would like to split into two Datasets. One Dataset[Mary] and one Dataset[John]. I would like to have as many rows based the amount of structures that exist in the add_marry or add_john arrays. I want to use Datasets and therefore not use the Dataframe API which would allow me to explode. Any advice on this would be greatly appreciated! The input is</p>
  <pre><code>List(General(&quot;1&quot;,&quot;Bob&quot;,[{&quot;1&quot;,&quot;2&quot;},{&quot;1&quot;,&quot;3&quot;}],[{&quot;11&quot;,&quot;12&quot;},{&quot;21&quot;,&quot;23&quot;}])).toDS
  </code></pre>
  <p>And output would be</p>
  <pre><code>Dataset[John] = ((&quot;1&quot;,&quot;Bob&quot;,&quot;11&quot;,&quot;12&quot;),(&quot;1&quot;,&quot;Bob&quot;,&quot;21&quot;,&quot;12&quot;))
  Dataset[Mary] = ((&quot;1&quot;,&quot;Bob&quot;,&quot;1&quot;,&quot;2&quot;),(&quot;1&quot;,&quot;Bob&quot;,&quot;1&quot;,&quot;3&quot;))
  </code></pre>
  <p>Classes =</p>
  <pre><code>case class General (
   id: String,
   name: String,
   add_marry: Array[AddressMary],
   add_john: Array[AddressJohn]
  )
  
  case class AddressMary (
   streetUK: String,
   houseUK: String
  )
  
  case class AddressJohn (
  streetUS: String,
  houseUS: String
  )
  
  case class John (
   id: String,
   name: String,
   street: String,
   house: String)
  
  case class Mary (
   id: String,
   name: String,
   street: String,
   house: String)
  </code></pre>
  <p>I have tried to do a for loop but this doesnt seem to work as I am not getting a row for each address that they have.</p>
  <pre><code>val dsHome = dfHome.as[General]
        .flatMap(pf =&gt; {
  if (pf.add_marry.size &gt; 0) for (e &lt;- pf) {
          Mary(pf.id,pf.name,e.streetUK, e.houseUK)
        }
  </code></pre>