title: How to read field from nested json?
body: <p>this is my test json file.</p>
  <pre><code>{
    &quot;item&quot; : {
      &quot;fracData&quot; : [ ],
      &quot;fractimeData&quot; : [ {
        &quot;number&quot; : &quot;1232323232&quot;,
        &quot;timePeriods&quot; : [ {
          &quot;validFrom&quot; : &quot;2021-08-03&quot;
        } ]
      } ],
      &quot;Module&quot; : [ ]
    }
  }
  </code></pre>
  <p>This is how I read the json file.</p>
  <pre><code>starhist_test_df = spark.read.json(&quot;/mapr/xxx/yyy/ttt/dev/rawdata/Test.json&quot;, multiLine=True)
  starhist_test_df.createOrReplaceTempView(&quot;v_test_df&quot;)
  </code></pre>
  <p>This query works.</p>
  <pre><code>df_test_01 = spark.sql(&quot;&quot;&quot;
   select item.fractimeData.number from v_test_df&quot;&quot;&quot;)
  df_test_01.collect();
  </code></pre>
  <p>Result</p>
  <pre><code>[Row(number=['1232323232'])]
  </code></pre>
  <p>But this query doesn't work.</p>
  <pre><code>df_test_01 = spark.sql(&quot;&quot;&quot;
   select item.fractimeData.timePeriods.validFrom from v_test_df&quot;&quot;&quot;)
  df_test_01.collect();
  </code></pre>
  <p>Error</p>
  <pre><code>cannot resolve 'v_test_df.`item`.`fractimeData`.`timePeriods`['validFrom']' due to data type mismatch: argument 2 requires integral type, however, ''validFrom'' is of string type.; line 3 pos 0;
  </code></pre>
  <p>What do I have to change, to read the validFrom field?</p>