{"action":"opened","issue":{"url":"https://api.github.com/repos/DataSystemsLab/GeoSpark/issues/277","repository_url":"https://api.github.com/repos/DataSystemsLab/GeoSpark","labels_url":"https://api.github.com/repos/DataSystemsLab/GeoSpark/issues/277/labels{/name}","comments_url":"https://api.github.com/repos/DataSystemsLab/GeoSpark/issues/277/comments","events_url":"https://api.github.com/repos/DataSystemsLab/GeoSpark/issues/277/events","html_url":"https://github.com/DataSystemsLab/GeoSpark/issues/277","id":367202091,"node_id":"MDU6SXNzdWUzNjcyMDIwOTE=","number":277,"title":"java.lang.ArrayIndexOutOfBoundsExceptionwhile reading shape file with partially empty user data","user":{"login":"tociek","id":6353615,"node_id":"MDQ6VXNlcjYzNTM2MTU=","avatar_url":"https://avatars0.githubusercontent.com/u/6353615?v=4","gravatar_id":"","url":"https://api.github.com/users/tociek","html_url":"https://github.com/tociek","followers_url":"https://api.github.com/users/tociek/followers","following_url":"https://api.github.com/users/tociek/following{/other_user}","gists_url":"https://api.github.com/users/tociek/gists{/gist_id}","starred_url":"https://api.github.com/users/tociek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tociek/subscriptions","organizations_url":"https://api.github.com/users/tociek/orgs","repos_url":"https://api.github.com/users/tociek/repos","events_url":"https://api.github.com/users/tociek/events{/privacy}","received_events_url":"https://api.github.com/users/tociek/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-10-05T13:03:20Z","updated_at":"2018-10-05T13:03:20Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"## Expected behavior\r\nWhen reading shape file, if non-spatial fields (user data) is an empty string, it should be loaded as empty string.\r\n## Actual behavior\r\nTested on shape file with two columns: spatial and id (string). When id column is empty, then geospark/spark thows java.lang.ArrayIndexOutOfBoundsException\r\n## Steps to reproduce the problem\r\nTry loading shape file with two columns. Non spatial column should be empty for some rows (happy to share sample)\r\n## Settings\r\ndefault\r\n\r\nGeoSpark version = 1.1.3\r\n\r\nApache Spark version = 2.1\r\n\r\nJRE version = 1.8? Yes\r\n\r\nAPI type = Scala or Java? Scala\r\n\r\nFull error:\r\n`scala> nhShapeDF.count\r\n18/10/05 08:55:05 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 31.0 (TID 45, azli-nhawonod111.pgazure.com, executor 11): java.lang.RuntimeException: Error while encoding: java.lang.ArrayIndexOutOfBoundsException: 1\r\nif (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true) AS rddshape#382\r\n+- if (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true)\r\n   :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt\r\n   :  :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n   :  :  +- input[0, org.apache.spark.sql.Row, true]\r\n   :  +- 0\r\n   :- null\r\n   +- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true)\r\n      +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType)\r\n         +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape)\r\n            +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n               +- input[0, org.apache.spark.sql.Row, true]\r\n\r\nif (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true) AS _c1#383\r\n+- if (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true)\r\n   :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt\r\n   :  :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n   :  :  +- input[0, org.apache.spark.sql.Row, true]\r\n   :  +- 1\r\n   :- null\r\n   +- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true)\r\n      +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType)\r\n         +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1)\r\n            +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n               +- input[0, org.apache.spark.sql.Row, true]\r\n\r\n        at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:293)\r\n        at org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:547)\r\n        at org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:547)\r\n        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)\r\n        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithoutKey$(Unknown Source)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)\r\n        at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)\r\n        at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)\r\n        at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)\r\n        at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)\r\n        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)\r\n        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)\r\n        at org.apache.spark.scheduler.Task.run(Task.scala:99)\r\n        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:748)\r\nCaused by: java.lang.ArrayIndexOutOfBoundsException: 1\r\n        at org.apache.spark.sql.catalyst.expressions.GenericRow.get(rows.scala:173)\r\n        at org.apache.spark.sql.Row$class.isNullAt(Row.scala:191)\r\n        at org.apache.spark.sql.catalyst.expressions.GenericRow.isNullAt(rows.scala:165)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.evalIfCondExpr1$(Unknown Source)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)\r\n        at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:290)\r\n        ... 17 more\r\n\r\n18/10/05 08:55:05 ERROR scheduler.TaskSetManager: Task 0 in stage 31.0 failed 4 times; aborting job\r\norg.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 31.0 failed 4 times, most recent failure: Lost task 0.3 in stage 31.0 (TID 48, azli-nhawonod111.pgazure.com, executor 11): java.lang.RuntimeException: Error while encoding: java.lang.ArrayIndexOutOfBoundsException: 1\r\nif (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true) AS rddshape#382\r\n+- if (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true)\r\n   :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt\r\n   :  :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n   :  :  +- input[0, org.apache.spark.sql.Row, true]\r\n   :  +- 0\r\n   :- null\r\n   +- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true)\r\n      +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType)\r\n         +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape)\r\n            +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n               +- input[0, org.apache.spark.sql.Row, true]\r\n\r\nif (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true) AS _c1#383\r\n+- if (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true)\r\n   :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt\r\n   :  :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n   :  :  +- input[0, org.apache.spark.sql.Row, true]\r\n   :  +- 1\r\n   :- null\r\n   +- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true)\r\n      +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType)\r\n         +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1)\r\n            +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n               +- input[0, org.apache.spark.sql.Row, true]\r\n\r\n        at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:293)\r\n        at org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:547)\r\n        at org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:547)\r\n        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)\r\n        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithoutKey$(Unknown Source)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)\r\n        at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)\r\n        at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)\r\n        at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)\r\n        at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)\r\n        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)\r\n        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)\r\n        at org.apache.spark.scheduler.Task.run(Task.scala:99)\r\n        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:748)\r\nCaused by: java.lang.ArrayIndexOutOfBoundsException: 1\r\n        at org.apache.spark.sql.catalyst.expressions.GenericRow.get(rows.scala:173)\r\n        at org.apache.spark.sql.Row$class.isNullAt(Row.scala:191)\r\n        at org.apache.spark.sql.catalyst.expressions.GenericRow.isNullAt(rows.scala:165)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.evalIfCondExpr1$(Unknown Source)\r\n        at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)\r\n        at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:290)\r\n        ... 17 more\r\n\r\nDriver stacktrace:\r\n  at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)\r\n  at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)\r\n  at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)\r\n  at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)\r\n  at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)\r\n  at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)\r\n  at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)\r\n  at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)\r\n  at scala.Option.foreach(Option.scala:257)\r\n  at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)\r\n  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)\r\n  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)\r\n  at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)\r\n  at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)\r\n  at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)\r\n  at org.apache.spark.SparkContext.runJob(SparkContext.scala:1918)\r\n  at org.apache.spark.SparkContext.runJob(SparkContext.scala:1931)\r\n  at org.apache.spark.SparkContext.runJob(SparkContext.scala:1944)\r\n  at org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)\r\n  at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:935)\r\n  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)\r\n  at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)\r\n  at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)\r\n  at org.apache.spark.rdd.RDD.collect(RDD.scala:934)\r\n  at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:275)\r\n  at org.apache.spark.sql.Dataset$$anonfun$org$apache$spark$sql$Dataset$$execute$1$1.apply(Dataset.scala:2371)\r\n  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:57)\r\n  at org.apache.spark.sql.Dataset.withNewExecutionId(Dataset.scala:2765)\r\n  at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$execute$1(Dataset.scala:2370)\r\n  at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$collect(Dataset.scala:2377)\r\n  at org.apache.spark.sql.Dataset$$anonfun$count$1.apply(Dataset.scala:2405)\r\n  at org.apache.spark.sql.Dataset$$anonfun$count$1.apply(Dataset.scala:2404)\r\n  at org.apache.spark.sql.Dataset.withCallback(Dataset.scala:2778)\r\n  at org.apache.spark.sql.Dataset.count(Dataset.scala:2404)\r\n  ... 48 elided\r\nCaused by: java.lang.RuntimeException: Error while encoding: java.lang.ArrayIndexOutOfBoundsException: 1\r\nif (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true) AS rddshape#382\r\n+- if (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true)\r\n   :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt\r\n   :  :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n   :  :  +- input[0, org.apache.spark.sql.Row, true]\r\n   :  +- 0\r\n   :- null\r\n   +- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType), true)\r\n      +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape), StringType)\r\n         +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 0, rddshape)\r\n            +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n               +- input[0, org.apache.spark.sql.Row, true]\r\n\r\nif (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true) AS _c1#383\r\n+- if (assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt) null else staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true)\r\n   :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object).isNullAt\r\n   :  :- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n   :  :  +- input[0, org.apache.spark.sql.Row, true]\r\n   :  +- 1\r\n   :- null\r\n   +- staticinvoke(class org.apache.spark.unsafe.types.UTF8String, StringType, fromString, validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType), true)\r\n      +- validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1), StringType)\r\n         +- getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object), 1, _c1)\r\n            +- assertnotnull(input[0, org.apache.spark.sql.Row, true], top level row object)\r\n               +- input[0, org.apache.spark.sql.Row, true]\r\n\r\n  at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:293)\r\n  at org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:547)\r\n  at org.apache.spark.sql.SparkSession$$anonfun$3.apply(SparkSession.scala:547)\r\n  at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)\r\n  at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)\r\n  at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithoutKey$(Unknown Source)\r\n  at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)\r\n  at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)\r\n  at org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)\r\n  at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)\r\n  at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:126)\r\n  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)\r\n  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)\r\n  at org.apache.spark.scheduler.Task.run(Task.scala:99)\r\n  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)\r\n  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n  at java.lang.Thread.run(Thread.java:748)\r\nCaused by: java.lang.ArrayIndexOutOfBoundsException: 1\r\n  at org.apache.spark.sql.catalyst.expressions.GenericRow.get(rows.scala:173)\r\n  at org.apache.spark.sql.Row$class.isNullAt(Row.scala:191)\r\n  at org.apache.spark.sql.catalyst.expressions.GenericRow.isNullAt(rows.scala:165)\r\n  at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.evalIfCondExpr1$(Unknown Source)\r\n  at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)\r\n  at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.toRow(ExpressionEncoder.scala:290)\r\n  ... 17 more\r\n`\r\n"}}
{"actor":{"display_login":"tociek"}}