title: Rearranging StrucType and nested array of struct
body: <p>I have a dataframe with the schema:</p>
  <pre class="lang-none prettyprint-override"><code>root
   |-- col2: integer (nullable = true)
   |-- col1: integer (nullable = true)
   |-- structCol3: struct (nullable = true)
   |    |-- structField2: boolean (nullable = true)
   |    |-- structField1: string (nullable = true)
   |-- structCol4: struct (nullable = true)
   |    |-- nestedArray: array (nullable = true)
   |    |    |-- element: struct (containsNull = true)
   |    |    |    |-- elem3: double (nullable = true)
   |    |    |    |-- elem2: string (nullable = true)
   |    |    |    |-- elem1: string (nullable = true)
   |    |-- structField2: integer (nullable = true)
  </code></pre>
  <p>Desired schema:</p>
  <pre class="lang-none prettyprint-override"><code>root
   |-- col1: integer (nullable = true) 
   |-- col2: integer (nullable = true)
   |-- structCol3: struct (nullable = true)
   |    |-- structField1: string (nullable = true)
   |    |-- structField2: boolean (nullable = true)
   |-- structCol4: struct (nullable = true)
   |    |-- nestedArray: array (nullable = true)
   |    |    |-- element: struct (containsNull = true)
   |    |    |    |-- elem1: string (nullable = true)
   |    |    |    |-- elem2: string (nullable = true)
   |    |    |    |-- elem3: double (nullable = true)
   |    |-- structField2: integer (nullable = true)
  </code></pre>
  <p>So far I've had success rearranging the columns and the fields inside the structs like this:</p>
  <pre class="lang-py prettyprint-override"><code>dfParquetOutput = df.select(
      &quot;col1&quot;,
      &quot;col2&quot;,
      struct(
          col(&quot;structCol3.structField1&quot;), 
          col(&quot;structCol3.structField2&quot;)
      ).alias(&quot;structCol3&quot;),
      struct(
          col(&quot;structCol4.nestedArray&quot;),
          col(&quot;structCol4.structField2&quot;)
      ).alias(&quot;structCol4&quot;)
  )
  </code></pre>
  <p>Unfortunately I'm struggling to find a way to re-arrange the elements inside the StructType that is inside the Array. I thought about trying to use a udf but I've had no success with it.</p>
  <p>Is there a simple way to re-ordering the Struct inside the array?</p>