{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/11752","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/11752/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/11752/comments","events_url":"https://api.github.com/repos/apache/spark/issues/11752/events","html_url":"https://github.com/apache/spark/pull/11752","id":141172063,"number":11752,"title":"[SPARK-3308][SQL][FOLLOW-UP] Parse JSON rows having an array type and a struct type in the same fieild","user":{"login":"HyukjinKwon","id":6477701,"avatar_url":"https://avatars.githubusercontent.com/u/6477701?v=3","gravatar_id":"","url":"https://api.github.com/users/HyukjinKwon","html_url":"https://github.com/HyukjinKwon","followers_url":"https://api.github.com/users/HyukjinKwon/followers","following_url":"https://api.github.com/users/HyukjinKwon/following{/other_user}","gists_url":"https://api.github.com/users/HyukjinKwon/gists{/gist_id}","starred_url":"https://api.github.com/users/HyukjinKwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HyukjinKwon/subscriptions","organizations_url":"https://api.github.com/users/HyukjinKwon/orgs","repos_url":"https://api.github.com/users/HyukjinKwon/repos","events_url":"https://api.github.com/users/HyukjinKwon/events{/privacy}","received_events_url":"https://api.github.com/users/HyukjinKwon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-03-16T04:46:01Z","updated_at":"2016-03-16T06:36:50Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/11752","html_url":"https://github.com/apache/spark/pull/11752","diff_url":"https://github.com/apache/spark/pull/11752.diff","patch_url":"https://github.com/apache/spark/pull/11752.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis https://github.com/apache/spark/pull/2400 added the support to parse JSON rows wrapped with an array. However, when the given data contains array data and struct data in the same field as below:\r\n\r\n```json\r\n{\"a\": {\"b\": 1}}\r\n{\"a\": []}\r\n```\r\n\r\nand the schema is given as below:\r\n\r\n```scala\r\nval schema =\r\n  StructType(\r\n    StructField(\"a\", StructType(\r\n      StructField(\"b\", StringType) :: Nil\r\n    )) :: Nil)\r\n```\r\n\r\nThis throws an exception as below:\r\n```java\r\nException in thread \"main\" org.apache.spark.SparkException: Job aborted due to stage failure: Task 7 in stage 0.0 failed 4 times, most recent failure: Lost task 7.3 in stage 0.0 (TID 10, 192.168.1.170): java.lang.ClassCastException: org.apache.spark.sql.types.GenericArrayData cannot be cast to org.apache.spark.sql.catalyst.InternalRow\r\n\tat org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getStruct(rows.scala:50)\r\n\tat org.apache.spark.sql.catalyst.expressions.GenericMutableRow.getStruct(rows.scala:247)\r\n\tat org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificPredicate.eval(Unknown Source)\r\n...\r\n```\r\n\r\nFor other data types, in this case it converts the given values are `null` but only this case emits an exception. \r\n\r\nThis PR makes the support for wrapped rows applied only at the top level.\r\n\r\n## How was this patch tested?\r\n\r\nUnit tests were used and `./dev/run_tests` for code style tests."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/197173995","html_url":"https://github.com/apache/spark/pull/11752#issuecomment-197173995","issue_url":"https://api.github.com/repos/apache/spark/issues/11752","id":197173995,"user":{"login":"HyukjinKwon","id":6477701,"avatar_url":"https://avatars.githubusercontent.com/u/6477701?v=3","gravatar_id":"","url":"https://api.github.com/users/HyukjinKwon","html_url":"https://github.com/HyukjinKwon","followers_url":"https://api.github.com/users/HyukjinKwon/followers","following_url":"https://api.github.com/users/HyukjinKwon/following{/other_user}","gists_url":"https://api.github.com/users/HyukjinKwon/gists{/gist_id}","starred_url":"https://api.github.com/users/HyukjinKwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HyukjinKwon/subscriptions","organizations_url":"https://api.github.com/users/HyukjinKwon/orgs","repos_url":"https://api.github.com/users/HyukjinKwon/repos","events_url":"https://api.github.com/users/HyukjinKwon/events{/privacy}","received_events_url":"https://api.github.com/users/HyukjinKwon/received_events","type":"User","site_admin":false},"created_at":"2016-03-16T06:36:50Z","updated_at":"2016-03-16T06:36:50Z","body":"Set `null` instead of throwing an exception."}}