title: Converting Array of Strings in Spark to byte array and retrieving it back using UDFs
body: <p>I am trying to convert Array of Strings to byte-array in Spark and re-converting the byte-array to Array of Strings.</p>
  
  <p>However, I am not getting the String array back as I intend. Here is the code - </p>
  
  <pre><code>// UDFs for converting Array[String] to byte array and get back Array[String] from byte array
  import com.fasterxml.jackson.module.scala.DefaultScalaModule
  import com.fasterxml.jackson.databind.ObjectMapper 
  
  val mapper: ObjectMapper = new ObjectMapper
  mapper.registerModule(DefaultScalaModule)
  
  val convertToByteArray = udf((map: Seq[String]) =&gt; mapper.writeValueAsBytes(map))
  val convertToString = udf((a: Array[Byte])=&gt; new String(a))
  
  val arrayDF = Seq(
    ("x100", Array("p1","p2","p3","p4"))
  ).toDF("id", "myarray")
  arrayDF.printSchema()
  root
   |-- id: string (nullable = true)
   |-- myarray: array (nullable = true)
   |    |-- element: string (containsNull = true)
  arrayDF.show(false)
  +----+----------------+
  |id  |myarray         |
  +----+----------------+
  |x100|[p1, p2, p3, p4]|
  +----+----------------+
  
  val converted = arrayDF.withColumn("bytearray", convertToByteArray($"myarray")).select($"id",$"bytearray")
  converted.printSchema()
  root
   |-- id: string (nullable = true)
   |-- bytearray: binary (nullable = true)
  converted.show(false)
  +----+----------------------------------------------------------------+
  |id  |bytearray                                                       |
  +----+----------------------------------------------------------------+
  |x100|[5B 22 70 31 22 2C 22 70 32 22 2C 22 70 33 22 2C 22 70 34 22 5D]|
  +----+----------------------------------------------------------------+
  
  val getBack = converted.withColumn("getstring", convertToString($"bytearray")) 
  getBack.printSchema()
  root
   |-- id: string (nullable = true)
   |-- bytearray: binary (nullable = true)
   |-- getstring: string (nullable = true)
  getBack.show(false)
  +----+----------------------------------------------------------------+---------------------+
  |id  |bytearray                                                       |getstring            |
  +----+----------------------------------------------------------------+---------------------+
  |x100|[5B 22 70 31 22 2C 22 70 32 22 2C 22 70 33 22 2C 22 70 34 22 5D]|["p1","p2","p3","p4"]|
  +----+----------------------------------------------------------------+---------------------+
  </code></pre>
  
  <p>However, I want my final results as -</p>
  
  <pre><code>+----+----------------------------------------------------------------+---------------------+
  |id  |bytearray                                                       |getstring            |
  +----+----------------------------------------------------------------+---------------------+
  |x100|[5B 22 70 31 22 2C 22 70 32 22 2C 22 70 33 22 2C 22 70 34 22 5D]|[p1,p2,p3,p4]|
  +----+----------------------------------------------------------------+---------------------+
  </code></pre>
  
  <p>Here is the <code>pom.xml</code> that I use for creating byte-array</p>
  
  <pre><code>&lt;dependency&gt;
      &lt;groupId&gt;com.fasterxml.jackson.core&lt;/groupId&gt;
      &lt;artifactId&gt;jackson-core&lt;/artifactId&gt;
      &lt;version&gt;2.9.5&lt;/version&gt;
  &lt;/dependency&gt;
  </code></pre>