title: TypeError: a float is required pyspark
body: <p>I aim to calculate the haversine distance of the lat-long data ,I have used uci dataset for the same <a href="https://archive.ics.uci.edu/ml/datasets/GPS+Trajectories" rel="nofollow noreferrer">https://archive.ics.uci.edu/ml/datasets/GPS+Trajectories</a> (go_track_trackspoints.csv)
  I have used below code to calculate the distance</p>
  
  <pre><code>def dist(lon2, lat2,lon1, lat1):
              phi_1=toRadians(lat1)
              phi_2=toRadians(lat2)
              delta_phi=toRadians(lat2-lat1)
              delta_lambda=toRadians(lon2-lon1)
  
  
              a=sin(delta_phi/2.0)**2+cos(phi_1)*cos(phi_2)*sin(delta_lambda/2.0)**2
              c=2*atan2(sqrt(abs(a)),sqrt(abs((1-a))))
              return c * 6372.8
  </code></pre>
  
  <p>And Schema is </p>
  
  <pre><code>root
   |-- id: string (nullable = true)
   |-- latitude: string (nullable = true)
   |-- longitude: string (nullable = true)
   |-- track_id: string (nullable = true)
   |-- time: string (nullable = true)
  </code></pre>
  
  <p>I have loaded the data into a spark dataframe (gps_data)</p>
  
  <pre><code> +---+-----------------+-----------------+--------+-------------------+
      | id|         latitude|        longitude|track_id|               time|
      +---+-----------------+-----------------+--------+-------------------+
      |  1|-10.9393413858164|-37.0627421097422|       1|2014-09-13 07:24:32|
      |  2| -10.939341385769|-37.0627421097809|       1|2014-09-13 07:24:37|
      |  3|-10.9393239478718|-37.0627645137212|       1|2014-09-13 07:24:42|
      |  4|-10.9392105616561|-37.0628430455445|       1|2014-09-13 07:24:47|
      +---+-----------------+-----------------+--------+-------------------+
  </code></pre>
  
  <p>Using the below command I am trying to get a column of distance </p>
  
  <pre><code>my_window = Window.partitionBy().orderBy("time")  
  gps_d=gps_data.withColumn("dist", dist(
          "longitude", "latitude",
          lag("longitude", 1).over(my_window), lag("latitude", 1).over(my_window)
      ).alias("dist"))
  </code></pre>
  
  <p>But I am not able to solve an error and not able to find a solution either .Please help me!</p>
  
  <p>Error is :</p>
  
  <pre><code>---------------------------------------------------------------------------
  TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-219-6352041ff223&gt; in &lt;module&gt;()
        1 gps_d=gps_data.withColumn("dist", dist(
        2     "longitude", "latitude",
  ----&gt; 3     lag("longitude", 1).over(my_window), lag("latitude", 1).over(my_window)
        4 ).alias("dist"))
  
  &lt;ipython-input-218-2f781ab3b2fb&gt; in dist(lon2, lat2, lon1, lat1)
        9 
       10         a=sin(delta_phi/2.0)**2+cos(phi_1)*cos(phi_2)*sin(delta_lambda/2.0)**2
  ---&gt; 11         c=2*atan2(sqrt(a),sqrt(1-a))
       12         return c * 6372.8
  
  TypeError: a float is required
  </code></pre>
  
  <p>PS: I have checked that none of the column is null when loading csv to dataframe</p>