title: How to set partition for Window function for PySpark?
body: <p>I'm running a PySpark job, and I'm getting the following message:</p>
  
  <pre><code>WARN org.apache.spark.sql.execution.Window: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
  </code></pre>
  
  <p>What does the message indicate, and how do I define a partition for a Window operation?</p>
  
  <p>EDIT:</p>
  
  <p>I'm trying to rank on an entire column.</p>
  
  <p>My data is organized as:</p>
  
  <pre><code>A
  B
  A
  C
  D
  </code></pre>
  
  <p>And I want:</p>
  
  <pre><code>A,1
  B,3
  A,1
  C,4
  D,5
  </code></pre>
  
  <p>I don't think there should by a .partitionBy() for this, only .orderBy(). The trouble is, this appears to cause performance degradation. Is there another way to achieve this without a Window function?</p>
  
  <p>If I partition by the first column, the result would be:</p>
  
  <pre><code>A,1
  B,1
  A,1
  C,1
  D,1
  </code></pre>
  
  <p>Which I do not want.</p>