title: How to extract an element from a array in pyspark
body: <p>I have a data frame with following type:</p>
  <pre><code>col1|col2|col3|col4
  xxxx|yyyy|zzzz|[1111],[2222]
  </code></pre>
  <p>I want my output to be following type:</p>
  <pre><code>col1|col2|col3|col4|col5
  xxxx|yyyy|zzzz|1111|2222
  </code></pre>
  <p>My col4 is an array and I want to convert it to a separate column. What needs to be done?</p>
  <p>I saw many answers with <code>flatMap</code>, but they are increasing a row, I want just the tuple to be put in another column but in the same row</p>
  <p>The following is my actual schema:</p>
  <pre><code>root
   |-- PRIVATE_IP: string (nullable = true)
   |-- PRIVATE_PORT: integer (nullable = true)
   |-- DESTINATION_IP: string (nullable = true)
   |-- DESTINATION_PORT: integer (nullable = true)
   |-- collect_set(TIMESTAMP): array (nullable = true)
   |    |-- element: string (containsNull = true)
  </code></pre>
  <p>Also, can please someone help me with explanation on both dataframes and RDD's.</p>