{"action":"opened","number":1206,"pull_request":{"url":"https://api.github.com/repos/apache/drill/pulls/1206","id":180761241,"html_url":"https://github.com/apache/drill/pull/1206","diff_url":"https://github.com/apache/drill/pull/1206.diff","patch_url":"https://github.com/apache/drill/pull/1206.patch","issue_url":"https://api.github.com/repos/apache/drill/issues/1206","number":1206,"state":"open","locked":false,"title":"DRILL_6314: Add complex types to result set loader","user":{"login":"paul-rogers","id":3248881,"avatar_url":"https://avatars0.githubusercontent.com/u/3248881?v=4","gravatar_id":"","url":"https://api.github.com/users/paul-rogers","html_url":"https://github.com/paul-rogers","followers_url":"https://api.github.com/users/paul-rogers/followers","following_url":"https://api.github.com/users/paul-rogers/following{/other_user}","gists_url":"https://api.github.com/users/paul-rogers/gists{/gist_id}","starred_url":"https://api.github.com/users/paul-rogers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paul-rogers/subscriptions","organizations_url":"https://api.github.com/users/paul-rogers/orgs","repos_url":"https://api.github.com/users/paul-rogers/repos","events_url":"https://api.github.com/users/paul-rogers/events{/privacy}","received_events_url":"https://api.github.com/users/paul-rogers/received_events","type":"User","site_admin":false},"body":"This PR is a bit of a large one as it adds Union, (non repeated) List and Repeated List type support to the column accessors, row set abstraction, result set loader abstraction, and associated mechanisms. The good new is that, after this PR, all the row set and result set loader work will be complete; we'll then move onto the scan operator and readers.\r\n\r\nBoth Union and (non-repeated) List have very odd semantics that required some creative gyrations in the existing code. A (non-repeated) List can old a single type (List of VarChar) say, in which the list entries can be null to model a JSON list:\r\n```\r\n{a: [\"foo\", \"bar\"]} {a: null}\r\n```\r\n\r\nList entries can also be unions (which can include null values.) A List starts as a simple list (one type), then gets \"promoted\" to a Union. Much complexity was needed to hide this process behind the simple row set abstractions.\r\n\r\nThere is similarity between List and Union, between List, Repeated List and \"normal\" Repeated (array) types. Refactoring reflects these commonalities.\r\n\r\nDue to the complexity of the added types, this PR revises the mechanisms that build a row set from an existing schema. or a schema from a container.\r\n\r\nThis PR also includes a somewhat orthogonal projection mechanism that implements projection at the row set mechanism for simple columns, array values and elements within maps. This code is closely intertwined with schema creation, and it was not worth the effort to tease the two apart into separate PRs. \r\n\r\nExtensive unit tests show the results in action. These are probably the best place to start to understand the client view of the new mechanisms.\r\n\r\nThe work is divided up in a number of commits to help sort out work to each layer.\r\n\r\nThe row set mechanism is fully described [here](https://github.com/paul-rogers/drill/wiki/Batch-Handling-Upgrades).\r\n\r\nRather than write a long description here, please take a look at the code and the Wiki post. Then, post questions (specific or general) and I'll address those particular topics which need additional clarification.","created_at":"2018-04-11T00:24:28Z","updated_at":"2018-04-11T00:24:28Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/drill/pulls/1206/commits","review_comments_url":"https://api.github.com/repos/apache/drill/pulls/1206/comments","review_comment_url":"https://api.github.com/repos/apache/drill/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/drill/issues/1206/comments","statuses_url":"https://api.github.com/repos/apache/drill/statuses/655779f558d0a0cc36fd3a0a23a9c305b5adf521","head":{"label":"paul-rogers:DRILL-6314","ref":"DRILL-6314","sha":"655779f558d0a0cc36fd3a0a23a9c305b5adf521","user":{"login":"paul-rogers","id":3248881,"avatar_url":"https://avatars0.githubusercontent.com/u/3248881?v=4","gravatar_id":"","url":"https://api.github.com/users/paul-rogers","html_url":"https://github.com/paul-rogers","followers_url":"https://api.github.com/users/paul-rogers/followers","following_url":"https://api.github.com/users/paul-rogers/following{/other_user}","gists_url":"https://api.github.com/users/paul-rogers/gists{/gist_id}","starred_url":"https://api.github.com/users/paul-rogers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paul-rogers/subscriptions","organizations_url":"https://api.github.com/users/paul-rogers/orgs","repos_url":"https://api.github.com/users/paul-rogers/repos","events_url":"https://api.github.com/users/paul-rogers/events{/privacy}","received_events_url":"https://api.github.com/users/paul-rogers/received_events","type":"User","site_admin":false},"repo":{"id":55184848,"name":"drill","full_name":"paul-rogers/drill","owner":{"login":"paul-rogers","id":3248881,"avatar_url":"https://avatars0.githubusercontent.com/u/3248881?v=4","gravatar_id":"","url":"https://api.github.com/users/paul-rogers","html_url":"https://github.com/paul-rogers","followers_url":"https://api.github.com/users/paul-rogers/followers","following_url":"https://api.github.com/users/paul-rogers/following{/other_user}","gists_url":"https://api.github.com/users/paul-rogers/gists{/gist_id}","starred_url":"https://api.github.com/users/paul-rogers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paul-rogers/subscriptions","organizations_url":"https://api.github.com/users/paul-rogers/orgs","repos_url":"https://api.github.com/users/paul-rogers/repos","events_url":"https://api.github.com/users/paul-rogers/events{/privacy}","received_events_url":"https://api.github.com/users/paul-rogers/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/paul-rogers/drill","description":"Mirror of Apache Drill","fork":true,"url":"https://api.github.com/repos/paul-rogers/drill","forks_url":"https://api.github.com/repos/paul-rogers/drill/forks","keys_url":"https://api.github.com/repos/paul-rogers/drill/keys{/key_id}","collaborators_url":"https://api.github.com/repos/paul-rogers/drill/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/paul-rogers/drill/teams","hooks_url":"https://api.github.com/repos/paul-rogers/drill/hooks","issue_events_url":"https://api.github.com/repos/paul-rogers/drill/issues/events{/number}","events_url":"https://api.github.com/repos/paul-rogers/drill/events","assignees_url":"https://api.github.com/repos/paul-rogers/drill/assignees{/user}","branches_url":"https://api.github.com/repos/paul-rogers/drill/branches{/branch}","tags_url":"https://api.github.com/repos/paul-rogers/drill/tags","blobs_url":"https://api.github.com/repos/paul-rogers/drill/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/paul-rogers/drill/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/paul-rogers/drill/git/refs{/sha}","trees_url":"https://api.github.com/repos/paul-rogers/drill/git/trees{/sha}","statuses_url":"https://api.github.com/repos/paul-rogers/drill/statuses/{sha}","languages_url":"https://api.github.com/repos/paul-rogers/drill/languages","stargazers_url":"https://api.github.com/repos/paul-rogers/drill/stargazers","contributors_url":"https://api.github.com/repos/paul-rogers/drill/contributors","subscribers_url":"https://api.github.com/repos/paul-rogers/drill/subscribers","subscription_url":"https://api.github.com/repos/paul-rogers/drill/subscription","commits_url":"https://api.github.com/repos/paul-rogers/drill/commits{/sha}","git_commits_url":"https://api.github.com/repos/paul-rogers/drill/git/commits{/sha}","comments_url":"https://api.github.com/repos/paul-rogers/drill/comments{/number}","issue_comment_url":"https://api.github.com/repos/paul-rogers/drill/issues/comments{/number}","contents_url":"https://api.github.com/repos/paul-rogers/drill/contents/{+path}","compare_url":"https://api.github.com/repos/paul-rogers/drill/compare/{base}...{head}","merges_url":"https://api.github.com/repos/paul-rogers/drill/merges","archive_url":"https://api.github.com/repos/paul-rogers/drill/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/paul-rogers/drill/downloads","issues_url":"https://api.github.com/repos/paul-rogers/drill/issues{/number}","pulls_url":"https://api.github.com/repos/paul-rogers/drill/pulls{/number}","milestones_url":"https://api.github.com/repos/paul-rogers/drill/milestones{/number}","notifications_url":"https://api.github.com/repos/paul-rogers/drill/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/paul-rogers/drill/labels{/name}","releases_url":"https://api.github.com/repos/paul-rogers/drill/releases{/id}","deployments_url":"https://api.github.com/repos/paul-rogers/drill/deployments","created_at":"2016-03-31T21:42:23Z","updated_at":"2018-04-08T17:29:27Z","pushed_at":"2018-04-10T22:20:17Z","git_url":"git://github.com/paul-rogers/drill.git","ssh_url":"git@github.com:paul-rogers/drill.git","clone_url":"https://github.com/paul-rogers/drill.git","svn_url":"https://github.com/paul-rogers/drill","homepage":"","size":57601,"stargazers_count":3,"watchers_count":3,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":1,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":0,"open_issues":1,"watchers":3,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"da241134fb88464139437b05b1feaafbb3014bb0","user":{"login":"apache","id":47359,"avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":5683653,"name":"drill","full_name":"apache/drill","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/drill","description":"Mirror of Apache Drill","fork":false,"url":"https://api.github.com/repos/apache/drill","forks_url":"https://api.github.com/repos/apache/drill/forks","keys_url":"https://api.github.com/repos/apache/drill/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/drill/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/drill/teams","hooks_url":"https://api.github.com/repos/apache/drill/hooks","issue_events_url":"https://api.github.com/repos/apache/drill/issues/events{/number}","events_url":"https://api.github.com/repos/apache/drill/events","assignees_url":"https://api.github.com/repos/apache/drill/assignees{/user}","branches_url":"https://api.github.com/repos/apache/drill/branches{/branch}","tags_url":"https://api.github.com/repos/apache/drill/tags","blobs_url":"https://api.github.com/repos/apache/drill/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/drill/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/drill/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/drill/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/drill/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/drill/languages","stargazers_url":"https://api.github.com/repos/apache/drill/stargazers","contributors_url":"https://api.github.com/repos/apache/drill/contributors","subscribers_url":"https://api.github.com/repos/apache/drill/subscribers","subscription_url":"https://api.github.com/repos/apache/drill/subscription","commits_url":"https://api.github.com/repos/apache/drill/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/drill/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/drill/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/drill/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/drill/contents/{+path}","compare_url":"https://api.github.com/repos/apache/drill/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/drill/merges","archive_url":"https://api.github.com/repos/apache/drill/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/drill/downloads","issues_url":"https://api.github.com/repos/apache/drill/issues{/number}","pulls_url":"https://api.github.com/repos/apache/drill/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/drill/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/drill/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/drill/labels{/name}","releases_url":"https://api.github.com/repos/apache/drill/releases{/id}","deployments_url":"https://api.github.com/repos/apache/drill/deployments","created_at":"2012-09-05T07:00:26Z","updated_at":"2018-04-10T15:11:00Z","pushed_at":"2018-04-10T22:54:20Z","git_url":"git://github.com/apache/drill.git","ssh_url":"git@github.com:apache/drill.git","clone_url":"https://github.com/apache/drill.git","svn_url":"https://github.com/apache/drill","homepage":"","size":56111,"stargazers_count":938,"watchers_count":938,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":646,"mirror_url":null,"archived":false,"open_issues_count":146,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":646,"open_issues":146,"watchers":938,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/drill/pulls/1206"},"html":{"href":"https://github.com/apache/drill/pull/1206"},"issue":{"href":"https://api.github.com/repos/apache/drill/issues/1206"},"comments":{"href":"https://api.github.com/repos/apache/drill/issues/1206/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/drill/pulls/1206/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/drill/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/drill/pulls/1206/commits"},"statuses":{"href":"https://api.github.com/repos/apache/drill/statuses/655779f558d0a0cc36fd3a0a23a9c305b5adf521"}},"author_association":"CONTRIBUTOR","merged":false,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"maintainer_can_modify":true,"commits":6,"additions":11550,"deletions":2071,"changed_files":116}}
{"actor":{"display_login":"paul-rogers"}}