title: Why the max value is not correct from dataframe in pyspark?
body: <p>I have a dataframe named as orders, loaded from csv file, while the <em>days_since_prior_order</em> column has some blank values. </p>
  
  <pre><code>orders = spark.read.csv("/Users/yanan.chen/Downloads/instacart/orders.csv",header=True)
  orders.createOrReplaceTempView("orders")
  spark.sql("select * from orders limit 30").show()
  
  +--------+-------+--------+------------+---------+-----------------+----------------------+
  |order_id|user_id|eval_set|order_number|order_dow|order_hour_of_day|days_since_prior_order|
  +--------+-------+--------+------------+---------+-----------------+----------------------+
  | 2539329|      1|   prior|           1|        2|               08|                      |
  | 2398795|      1|   prior|           2|        3|               07|                  15.0|
  |  473747|      1|   prior|           3|        3|               12|                  21.0|
  | 2254736|      1|   prior|           4|        4|               07|                  29.0|
  |  431534|      1|   prior|           5|        4|               15|                  28.0|
  | 3367565|      1|   prior|           6|        2|               07|                  19.0|
  |  550135|      1|   prior|           7|        1|               09|                  20.0|
  | 3108588|      1|   prior|           8|        1|               14|                  14.0|
  | 2295261|      1|   prior|           9|        1|               16|                   0.0|
  | 2550362|      1|   prior|          10|        4|               08|                  30.0|
  | 1187899|      1|   train|          11|        4|               08|                  14.0|
  | 2168274|      2|   prior|           1|        2|               11|                      |
  | 1501582|      2|   prior|           2|        5|               10|                  10.0|
  | 1901567|      2|   prior|           3|        1|               10|                   3.0|
  |  738281|      2|   prior|           4|        2|               10|                   8.0|
  | 1673511|      2|   prior|           5|        3|               11|                   8.0|
  | 1199898|      2|   prior|           6|        2|               09|                  13.0|
  | 3194192|      2|   prior|           7|        2|               12|                  14.0|
  |  788338|      2|   prior|           8|        1|               15|                  27.0|
  | 1718559|      2|   prior|           9|        2|               09|                   8.0|
  +--------+-------+--------+------------+---------+-----------------+----------------------+
  </code></pre>
  
  <p>As you can see, there are some blanks in <code>days_since_prior_order</code>, which as actually ''. <code>spark.sql("select * from orders where days_since_prior_order &lt;&gt; '' ").show()</code></p>
  
  <pre><code>+--------+-------+--------+------------+---------+-----------------+----------------------+
  |order_id|user_id|eval_set|order_number|order_dow|order_hour_of_day|days_since_prior_order|
  +--------+-------+--------+------------+---------+-----------------+----------------------+
  | 2398795|      1|   prior|           2|        3|               07|                  15.0|
  |  473747|      1|   prior|           3|        3|               12|                  21.0|
  | 2254736|      1|   prior|           4|        4|               07|                  29.0|
  |  431534|      1|   prior|           5|        4|               15|                  28.0|
  | 3367565|      1|   prior|           6|        2|               07|                  19.0|
  |  550135|      1|   prior|           7|        1|               09|                  20.0|
  | 3108588|      1|   prior|           8|        1|               14|                  14.0|
  | 2295261|      1|   prior|           9|        1|               16|                   0.0|
  | 2550362|      1|   prior|          10|        4|               08|                  30.0|
  | 1187899|      1|   train|          11|        4|               08|                  14.0|
  | 1501582|      2|   prior|           2|        5|               10|                  10.0|
  | 1901567|      2|   prior|           3|        1|               10|                   3.0|
  |  738281|      2|   prior|           4|        2|               10|                   8.0|
  | 1673511|      2|   prior|           5|        3|               11|                   8.0|
  | 1199898|      2|   prior|           6|        2|               09|                  13.0|
  | 3194192|      2|   prior|           7|        2|               12|                  14.0|
  |  788338|      2|   prior|           8|        1|               15|                  27.0|
  | 1718559|      2|   prior|           9|        2|               09|                   8.0|
  | 1447487|      2|   prior|          10|        1|               11|                   6.0|
  | 1402090|      2|   prior|          11|        1|               10|                  30.0|
  +--------+-------+--------+------------+---------+-----------------+----------------------+
  </code></pre>
  
  <p>But, what confused me is that when I excute <code>spark.sql("select min(days_since_prior_order),  max(days_since_prior_order) from orders where days_since_prior_order &lt;&gt; '' ").show()</code> The max value in result is not correct.</p>
  
  <pre><code>+---------------------------+---------------------------+
  |min(days_since_prior_order)|max(days_since_prior_order)|
  +---------------------------+---------------------------+
  |                        0.0|                        9.0|
  +---------------------------+---------------------------+
  </code></pre>
  
  <p>What's wrong with my codes ?</p>