title: How to generate id and put in pyspark df?
body: <p>I am reading data from a hive table and creating DataFrame in pyspark using:</p>
  
  <pre><code>hive_df = sqlContext.sql("select * from table") 
  </code></pre>
  
  <p>The DataFrame <code>hive_df</code> has three columns: <code>(cust_id, name, l_name)</code></p>
  
  <p>In <code>hive table</code> the <code>cust_id</code> field is <code>null</code> for all the records, so I want to put some value, in incremental manner.</p>
  
  <p><strong>Data in hive table</strong></p>
  
  <pre><code>cust_id,name,l_name
         , abc,   def
         , ghi,   jkl
         , mno,   pqr
  </code></pre>
  
  <p><strong>Desired Output</strong></p>
  
  <pre><code>cust_id,name,l_name
     1000, abc,   def
     1001, ghi,   jkl
     1002, mno,   pqr
  </code></pre>