title: Value and column operations in scala spark, how to use a value left of an operator with spark column?
body: <p>I am trying to do some basic operations with Columns and Doubles and I can't figure out how to do it without creating a UDF.</p>
  
  <pre><code>scala&gt; import org.apache.spark.sql.functions.col
  scala&gt; import spark.implicits._
  
  scala&gt; val df = Seq(("A", 1), ("B", 2), ("C", 3)).toDF("col1", "col2")
  df: org.apache.spark.sql.DataFrame = [col1: string, col2: int]
  </code></pre>
  
  <p>I want to find the reciprocal of col2, I would think to do that would be something like:</p>
  
  <pre><code>scala&gt; df.withColumn("col3", 1/col("col2")).show
  </code></pre>
  
  <p>But that give this error:</p>
  
  <pre><code>&lt;console&gt;:30: error: overloaded method value / with alternatives:
    (x: Double)Double &lt;and&gt;
    (x: Float)Float &lt;and&gt;
    (x: Long)Long &lt;and&gt;
    (x: Int)Int &lt;and&gt;
    (x: Char)Int &lt;and&gt;
    (x: Short)Int &lt;and&gt;
    (x: Byte)Int
   cannot be applied to (org.apache.spark.sql.Column)
         df.withColumn("col3", 1/col("col2")).show
  </code></pre>
  
  <p>Basically saying that you can't perform division (or any other operator) with a Double on the left hand side and a Column on the right. The only way I have been able to figure out how to do this is to create a UDF and apply it like this:</p>
  
  <pre><code>scala&gt; def reciprocal(x: Double) : Double = {1/x}
  reciprocal: (x: Double)Double
  
  scala&gt; val reciprocalUDF = spark.sqlContext.udf.register(
         "reciprocalUDF", reciprocal _)
  
  scala&gt; df.withColumn("col3", reciprocalUDF(col("col2"))).show
  +----+----+------------------+
  |col1|col2|              col3|
  +----+----+------------------+
  |   A|   1|               1.0|
  |   B|   2|               0.5|
  |   C|   3|0.3333333333333333|
  +----+----+------------------+
  </code></pre>
  
  <p>But really? Are UDFs the only way to do this sort of thing? I don't want to create a UDF every time I have to do some simple operation like division.</p>