{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/23253","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/23253/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/23253/comments","events_url":"https://api.github.com/repos/apache/spark/issues/23253/events","html_url":"https://github.com/apache/spark/pull/23253","id":388586022,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM2ODE0NjUz","number":23253,"title":"[SPARK-26303][SQL] Return partial results for bad JSON records","user":{"login":"MaxGekk","id":1580697,"node_id":"MDQ6VXNlcjE1ODA2OTc=","avatar_url":"https://avatars1.githubusercontent.com/u/1580697?v=4","gravatar_id":"","url":"https://api.github.com/users/MaxGekk","html_url":"https://github.com/MaxGekk","followers_url":"https://api.github.com/users/MaxGekk/followers","following_url":"https://api.github.com/users/MaxGekk/following{/other_user}","gists_url":"https://api.github.com/users/MaxGekk/gists{/gist_id}","starred_url":"https://api.github.com/users/MaxGekk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MaxGekk/subscriptions","organizations_url":"https://api.github.com/users/MaxGekk/orgs","repos_url":"https://api.github.com/users/MaxGekk/repos","events_url":"https://api.github.com/users/MaxGekk/events{/privacy}","received_events_url":"https://api.github.com/users/MaxGekk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2018-12-07T10:09:10Z","updated_at":"2018-12-09T19:15:23Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/23253","html_url":"https://github.com/apache/spark/pull/23253","diff_url":"https://github.com/apache/spark/pull/23253.diff","patch_url":"https://github.com/apache/spark/pull/23253.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nIn the PR, I propose to return partial results from JSON datasource and JSON functions in the PERMISSIVE mode if some of JSON fields are parsed and converted to desired types successfully. The changes are made only for `StructType`. Whole bad JSON records are placed into the corrupt column specified by the `columnNameOfCorruptRecord` option or SQL config.\r\n\r\nPartial results are not returned for malformed JSON input. \r\n\r\n## How was this patch tested?\r\n\r\nAdded new UT which checks converting JSON strings with one invalid and one valid field at the end of the string.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/445563271","html_url":"https://github.com/apache/spark/pull/23253#issuecomment-445563271","issue_url":"https://api.github.com/repos/apache/spark/issues/23253","id":445563271,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NTU2MzI3MQ==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-12-09T19:15:23Z","updated_at":"2018-12-09T19:15:23Z","author_association":"NONE","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99888/\nTest PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}