title: How to convert column values from string to decimal?
body: <p>I'm having a dataframe which contains a really big integer value, example:</p>
  
  <pre><code>42306810747081022358
  </code></pre>
  
  <p>When I've tried to convert it to long it was working in the Java but not under the spark envrironment, I was getting </p>
  
  <pre><code>   NumberFormatException: For input string("42306810747081022358")
  </code></pre>
  
  <p>Then I tried to convert it too Decimal (BigDecimal) value. Again, easily can do it in Java, but in Spark: 
       dframe.withColumn("c_number",col("c_a").cast(new DecimalType()));</p>
  
  <p>This way I don't get any exceptions, however I can see that all result values are null. </p>
  
  <p>I also tried to use UDF for this purpose but get the same results: </p>
  
  <pre><code>UDF1 cTransformer = new UDF1&lt;String, BigDecimal&gt;() {
          @Override
          public BigDecimal call(String aString) throws Exception {
              return new BigDecimal(aString);
          }
      };
  sqlContext.udf().register("cTransformer", cTransformer, new DecimalType());
  dframe = dframe.withColumn("c_number", callUDF("cTransformer", dframe.col("c_a"))); 
  </code></pre>
  
  <p>And here again all I'm getting is a column with all zeroes. </p>
  
  <p>How should I proceed?</p>