title: Deciles or other quantile rank for Pyspark column
body: <p>I have a pyspark DF with multiple numeric columns and I want to, for each column calculate the decile or other quantile rank for that row based on each variable.</p>
  
  <p>This is simple for pandas as we can create a new column for each variable using the qcut function to assign the value 0 to n-1 for 'q' as in pd.qcut(x,q=n).</p>
  
  <p>How can this be done in pyspark? I have tried the following but clearly the break points are not unique between these thirds. I want to get the lower 1/3 of the data assigned 1, the next 1/3 assigned 2 and the top 1/3 assigned 3. I want to be able to change this and perhaps use 1/10, 1/32 etc</p>
  
  <pre><code>w =  Window.partitionBy(data.var1).orderBy(data.var1)
  d2=df.select(
      "var1",
       ntile(3).over(w).alias("ntile3")
  )
  
  
  agged=d2.groupby('ntile3').agg(F.min("var1").alias("min_var1"),F.max("var1").alias("max_var1"),F.count('*'))
  agged.show()
  
  +------+--------+--------+--------+
  |ntile3|min_var1|max_var1|count(1)|
  +------+--------+--------+--------+
  |     1|     0.0|   210.0|  517037|
  |     3|     0.0|   206.0|  516917|
  |     2|     0.0|   210.0|  516962|
  +------+--------+--------+--------+
  </code></pre>