title: Create sequential unique id for each group
body: <p>I'm trying to find an equivalent for the following snippet (<a href="https://stackoverflow.com/a/67873779/6347629">reference</a>) to create unique id to every unique combination from two columns in PySpark.</p>
  <p>Pandas approach:</p>
  <pre class="lang-py prettyprint-override"><code>df['my_id'] = df.groupby(['foo', 'bar'], sort=False).ngroup() + 1
  </code></pre>
  <p>I tried the following, but it's creating more ids than required:</p>
  <pre class="lang-py prettyprint-override"><code>df = df.withColumn(&quot;my_id&quot;, F.row_number().over(Window.orderBy('foo', 'bar')))
  </code></pre>