title: Pyspark - remove duplicates from dataframe keeping the last appearance
body: <p>I'm trying to dedupe a spark dataframe leaving only the latest appearance.
  The duplication is in three variables:</p>
  
  <pre><code>NAME
  ID
  DOB
  </code></pre>
  
  <p>I succeeded in Pandas with the following:</p>
  
  <pre><code>df_dedupe = df.drop_duplicates(subset=['NAME','ID','DOB'], keep='last', inplace=False)
  </code></pre>
  
  <p>But in spark I tried the following:</p>
  
  <pre><code>df_dedupe = df.dropDuplicates(['NAME', 'ID', 'DOB'], keep='last')
  </code></pre>
  
  <p>I get this error:</p>
  
  <pre><code>TypeError: dropDuplicates() got an unexpected keyword argument 'keep'
  </code></pre>
  
  <p>Any ideas?</p>