title: Pyspark: add incremental counter to rows, for each different value
body: <p>I need to add a "row number" to a dataframe, but this "row number" must restart for each new value in a column.</p>
  
  <p>Let me show you an example:</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code>from pyspark.sql import SparkSession
  spark = SparkSession.builder.appName('test').getOrCreate()
  
  df = spark.createDataFrame([
      ('2018-01-01', 'John', 0),
      ('2018-01-01', 'Paul', 1),
      ('2018-01-08', 'Paul', 3),
      ('2018-01-08', 'Pete', 4),
      ('2018-01-08', 'John', 3),
      ('2018-01-15', 'Mary', 6),
      ('2018-01-15', 'Pete', 6),
      ('2018-01-15', 'John', 6),
      ('2018-01-15', 'Paul', 6),
  ], ['str_date', 'name', 'value'])
  
  # Convert str_date to date:
  df = df.withColumn('date', to_date(df['str_date'])) \
      .select(['date', 'name', 'value'])
  
  # Sort by name and date
  df.orderBy(['name', 'date']).show()
  
  ## +----------+----+-----+
  ## |      date|name|value|
  ## +----------+----+-----+
  ## |2018-01-01|John|    0|
  ## |2018-01-08|John|    3|
  ## |2018-01-15|John|    6|
  ## |2018-01-15|Mary|    6|
  ## |2018-01-01|Paul|    1|
  ## |2018-01-08|Paul|    3|
  ## |2018-01-15|Paul|    6|
  ## |2018-01-08|Pete|    4|
  ## |2018-01-15|Pete|    6|
  ## +----------+----+-----+
  </code></pre>
  
  <p>So, what I need is to add a new column with the number of the row for each <code>name</code>:</p>
  
  <pre class="lang-python prettyprint-override"><code># Expected result
  ## +----------+----+-----+------+
  ## |      date|name|value|rowNum|
  ## +----------+----+-----+------+
  ## |2018-01-01|John|    0|     1| &lt;- First row for 'John'
  ## |2018-01-08|John|    3|     2|
  ## |2018-01-15|John|    6|     3|
  ## |2018-01-15|Mary|    6|     1| &lt;- First row for 'Mary'
  ## |2018-01-01|Paul|    1|     1| &lt;- First row for 'Paul'
  ## |2018-01-08|Paul|    3|     2|
  ## |2018-01-15|Paul|    6|     3|
  ## |2018-01-08|Pete|    4|     1| &lt;- First row for 'Pete'
  ## |2018-01-15|Pete|    6|     2|
  ## +----------+----+-----+------+
  </code></pre>
  
  <p>I've been trying with the <code>Window</code> function, but I'm stuck. Can you please help me?</p>
  
  <p><strong>Notes:</strong></p>
  
  <ul>
  <li>It is guaranteed that the rows will be sorted (and, if they are not sorted, they will be sorted as part of the work pipeline)</li>
  <li>I'm using Spark 2.4.0</li>
  </ul>