title: How to compute the largest value in a column using withColumn?
body: <p>I'm trying to compute the largest value of the following DataFrame in Spark <strong>1.6.1</strong>:</p>
  
  
  
  <pre class="lang-scala prettyprint-override"><code>val df = sc.parallelize(Seq(1,2,3)).toDF("id")
  </code></pre>
  
  <p>A first approach would be to select the maximum value, and it works as expected:</p>
  
  <pre class="lang-scala prettyprint-override"><code>df.select(max($"id")).show
  </code></pre>
  
  <p>The second approach could be to use <code>withColumn</code> as follows:</p>
  
  <pre class="lang-scala prettyprint-override"><code>df.withColumn("max", max($"id")).show
  </code></pre>
  
  <p>But unfortunately it fails with the following error message:</p>
  
  <blockquote>
    <p>org.apache.spark.sql.AnalysisException: expression 'id' is neither
    present in the group by, nor is it an aggregate function. Add to group
    by or wrap in first() (or first_value) if you don't care which value
    you get.;</p>
  </blockquote>
  
  <p>How can I compute the maximum value in a <code>withColumn</code> function without any <code>Window</code> or <code>groupBy</code>? If not possible, how can I do it in this specific case using a <code>Window</code>?</p>