title: Spark Error:expected zero arguments for construction of ClassDict (for numpy.core.multiarray._reconstruct)
body: <p>I have a dataframe in Spark in which one of the columns contains an array.Now,I have written a separate UDF which converts the array to another array with distinct values in it only. See example below:</p>
  
  <p>Ex: <strong>[24,23,27,23]</strong> should get converted to <strong>[24, 23, 27]</strong>
  Code:</p>
  
  <pre><code>def uniq_array(col_array):
      x = np.unique(col_array)
      return x
  uniq_array_udf = udf(uniq_array,ArrayType(IntegerType()))
  
  Df3 = Df2.withColumn("age_array_unique",uniq_array_udf(Df2.age_array))
  </code></pre>
  
  <p>In the above code, <code>Df2.age_array</code> is the array on which I am applying the UDF to get a different column <code>"age_array_unique"</code> which should contain only unique values in the array.</p>
  
  <p>However, as soon as I run the command <code>Df3.show()</code>, I get the error:</p>
  
  <blockquote>
    <p>net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict (for numpy.core.multiarray._reconstruct)</p>
  </blockquote>
  
  <p>Can anyone please let me know why this is happening?</p>
  
  <p>Thanks!</p>