title: What is wrong with spark sql substring function?
body: <p>This should require no explanation. But could someone describe the logic behind the pos parameter of substring because I cannot make sense of this (Using Spark 2.1):</p>
  
  <pre><code>scala&gt; val df = Seq("abcdef").toDS()
  df: org.apache.spark.sql.Dataset[String] = [value: string]
  
  scala&gt; df.show
  +------+
  | value|
  +------+
  |abcdef|
  +------+
  
  scala&gt; df.selectExpr("substring(value, 0, 2)", "substring(value, 1, 2)", "substring(value, 2,2)", "substring(value, 3,2)").show
  +----------------------+----------------------+----------------------+----------------------+
  |substring(value, 0, 2)|substring(value, 1, 2)|substring(value, 2, 2)|substring(value, 3, 2)|
  +----------------------+----------------------+----------------------+----------------------+
  |                    ab|                    ab|                    bc|                    cd|
  +----------------------+----------------------+----------------------+----------------------+
  </code></pre>