{"action":"closed","number":5140,"pull_request":{"url":"https://api.github.com/repos/arangodb/arangodb/pulls/5140","id":182506744,"html_url":"https://github.com/arangodb/arangodb/pull/5140","diff_url":"https://github.com/arangodb/arangodb/pull/5140.diff","patch_url":"https://github.com/arangodb/arangodb/pull/5140.patch","issue_url":"https://api.github.com/repos/arangodb/arangodb/issues/5140","number":5140,"state":"closed","locked":false,"title":"Feature 3.3/aql optimizations","user":{"login":"jsteemann","id":1423118,"avatar_url":"https://avatars0.githubusercontent.com/u/1423118?v=4","gravatar_id":"","url":"https://api.github.com/users/jsteemann","html_url":"https://github.com/jsteemann","followers_url":"https://api.github.com/users/jsteemann/followers","following_url":"https://api.github.com/users/jsteemann/following{/other_user}","gists_url":"https://api.github.com/users/jsteemann/gists{/gist_id}","starred_url":"https://api.github.com/users/jsteemann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsteemann/subscriptions","organizations_url":"https://api.github.com/users/jsteemann/orgs","repos_url":"https://api.github.com/users/jsteemann/repos","events_url":"https://api.github.com/users/jsteemann/events{/privacy}","received_events_url":"https://api.github.com/users/jsteemann/received_events","type":"User","site_admin":false},"body":"Adds the following optimizations:\r\n\r\n* remove post-sort from GatherNode in cluster AQL queries that do use indexes\r\n  for filtering but that do not require a sorted result\r\n\r\n  This optimization can speed up gathering data from multiple shards, because\r\n  it allows to remove a merge sort of the individual shards' results.\r\n\r\n* extend the already existing \"reduce-extraction-to-projection\" AQL optimizer \r\n  rule for RocksDB to provide projections of up to 5 document attributes. The\r\n  previous implementation only supported a projection for a single document \r\n  attribute. The new implementation will extract up to 5 document attributes from \r\n  a document while scanning a collection via an EnumerateCollectionNode.\r\n  Additionally the new version of the optimizer rule can also produce projections\r\n  when scanning an index via an IndexNode. \r\n  The optimization is benefial especially for huge documents because it will copy\r\n  out only the projected attributes from the document instead of copying the entire \r\n  document data from the storage engine. \r\n\r\n  When applied, the explainer will show the projected attributes in a `projections`\r\n  remark for an EnumerateCollectionNode or IndexNode. The optimization is limited\r\n  to the RocksDB storage engine.\r\n\r\n* added index-only optimization for AQL queries that can satisfy the retrieval of\r\n  all required document attributes directly from an index.\r\n\r\n  This optimization will be triggered for the RocksDB engine if an index is used\r\n  that covers all required attributes of the document used later on in the query.\r\n  If applied, it will save retrieving the actual document data (which would require\r\n  an extra lookup in RocksDB), but will instead build the document data solely \r\n  from the index values found. It will only be applied when using up to 5 attributes\r\n  from the document, and only if the rest of the document data is not used later\r\n  on in the query.\r\n\r\n  The optimization is currently available for the RocksDB engine for the index types\r\n  primary, edge, hash, skiplist and persistent.\r\n\r\n  If the optimization is applied, it will show up as \"index only\" in an AQL\r\n  query's execution plan for an IndexNode.\r\n\r\n* added scan-only optimization for AQL queries that iterate over collections or\r\n  indexes and that do not need to return the actual document values.\r\n\r\n  Not fetching the document values from the storage engine will provide a \r\n  considerable speedup when using the RocksDB engine, but may also help a bit\r\n  in case of the MMFiles engine. The optimization will only be applied when \r\n  full-scanning or index-scanning a collection without refering to any of its\r\n  documents later on, and, for an IndexNode, if all filter conditions for the \r\n  documents of the collection are covered by the index.\r\n\r\n  If the optimization is applied, it will show up as \"scan only\" in an AQL\r\n  query's execution plan for an EnumerateCollectionNode or an IndexNode.\r\n\r\n* extend existing \"collect-in-cluster\" optimizer rule to run grouping, counting\r\n  and deduplication on the DB servers in several cases, so that the coordinator\r\n  will only need to sum up the potentially smaller results from the individual shards.\r\n\r\n  The following types of COLLECT queries are covered now:\r\n  - RETURN DISTINCT expr\r\n  - COLLECT WITH COUNT INTO ...\r\n  - COLLECT var1 = expr1, ..., varn = exprn (WITH COUNT INTO ...), without INTO or KEEP\r\n  - COLLECT var1 = expr1, ..., varn = exprn AGGREGATE ..., without INTO or KEEP, for\r\n    aggregate functions COUNT/LENGTH, SUM, MIN and MAX.\r\n  \r\n* honor specified COLLECT method in AQL COLLECT options\r\n\r\n  for example, when the user explicitly asks for the COLLECT method\r\n  to be `sorted`, the optimizer will now not produce an alternative\r\n  version of the plan using the hash method.\r\n\r\n  additionally, if the user explcitly asks for the COLLECT method to\r\n  be `hash`, the optimizer will now change the existing plan to use\r\n  the hash method if possible instead of just creating an alternative\r\n  plan.\r\n\r\n  `COLLECT ... OPTIONS { method: 'sorted' }` => always use sorted method\r\n  `COLLECT ... OPTIONS { method: 'hash' }`   => use hash if this is technically possible\r\n  `COLLECT ...` (no options)                 => create a plan using sorted, and another plan using hash method\r\n\r\nNote that there is also a corresponding enterprise branch.","created_at":"2018-04-18T14:54:01Z","updated_at":"2018-05-15T12:41:08Z","closed_at":"2018-05-15T12:41:08Z","merged_at":"2018-05-15T12:41:08Z","merge_commit_sha":"731925080d822b3ae565cd16129eac8b0d900d91","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":{"url":"https://api.github.com/repos/arangodb/arangodb/milestones/70","html_url":"https://github.com/arangodb/arangodb/milestone/70","labels_url":"https://api.github.com/repos/arangodb/arangodb/milestones/70/labels","id":2784823,"number":70,"title":"3.3","description":null,"creator":{"login":"dothebart","id":5013077,"avatar_url":"https://avatars1.githubusercontent.com/u/5013077?v=4","gravatar_id":"","url":"https://api.github.com/users/dothebart","html_url":"https://github.com/dothebart","followers_url":"https://api.github.com/users/dothebart/followers","following_url":"https://api.github.com/users/dothebart/following{/other_user}","gists_url":"https://api.github.com/users/dothebart/gists{/gist_id}","starred_url":"https://api.github.com/users/dothebart/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dothebart/subscriptions","organizations_url":"https://api.github.com/users/dothebart/orgs","repos_url":"https://api.github.com/users/dothebart/repos","events_url":"https://api.github.com/users/dothebart/events{/privacy}","received_events_url":"https://api.github.com/users/dothebart/received_events","type":"User","site_admin":false},"open_issues":7,"closed_issues":249,"state":"open","created_at":"2017-09-22T08:54:53Z","updated_at":"2018-05-15T12:41:08Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/arangodb/arangodb/pulls/5140/commits","review_comments_url":"https://api.github.com/repos/arangodb/arangodb/pulls/5140/comments","review_comment_url":"https://api.github.com/repos/arangodb/arangodb/pulls/comments{/number}","comments_url":"https://api.github.com/repos/arangodb/arangodb/issues/5140/comments","statuses_url":"https://api.github.com/repos/arangodb/arangodb/statuses/227af8f4d89739ffe0e084335e823f17ef405207","head":{"label":"arangodb:feature-3.3/aql-optimizations","ref":"feature-3.3/aql-optimizations","sha":"227af8f4d89739ffe0e084335e823f17ef405207","user":{"login":"arangodb","id":5547849,"avatar_url":"https://avatars0.githubusercontent.com/u/5547849?v=4","gravatar_id":"","url":"https://api.github.com/users/arangodb","html_url":"https://github.com/arangodb","followers_url":"https://api.github.com/users/arangodb/followers","following_url":"https://api.github.com/users/arangodb/following{/other_user}","gists_url":"https://api.github.com/users/arangodb/gists{/gist_id}","starred_url":"https://api.github.com/users/arangodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arangodb/subscriptions","organizations_url":"https://api.github.com/users/arangodb/orgs","repos_url":"https://api.github.com/users/arangodb/repos","events_url":"https://api.github.com/users/arangodb/events{/privacy}","received_events_url":"https://api.github.com/users/arangodb/received_events","type":"Organization","site_admin":false},"repo":{"id":2649214,"name":"arangodb","full_name":"arangodb/arangodb","owner":{"login":"arangodb","id":5547849,"avatar_url":"https://avatars0.githubusercontent.com/u/5547849?v=4","gravatar_id":"","url":"https://api.github.com/users/arangodb","html_url":"https://github.com/arangodb","followers_url":"https://api.github.com/users/arangodb/followers","following_url":"https://api.github.com/users/arangodb/following{/other_user}","gists_url":"https://api.github.com/users/arangodb/gists{/gist_id}","starred_url":"https://api.github.com/users/arangodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arangodb/subscriptions","organizations_url":"https://api.github.com/users/arangodb/orgs","repos_url":"https://api.github.com/users/arangodb/repos","events_url":"https://api.github.com/users/arangodb/events{/privacy}","received_events_url":"https://api.github.com/users/arangodb/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/arangodb/arangodb","description":"🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.","fork":false,"url":"https://api.github.com/repos/arangodb/arangodb","forks_url":"https://api.github.com/repos/arangodb/arangodb/forks","keys_url":"https://api.github.com/repos/arangodb/arangodb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/arangodb/arangodb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/arangodb/arangodb/teams","hooks_url":"https://api.github.com/repos/arangodb/arangodb/hooks","issue_events_url":"https://api.github.com/repos/arangodb/arangodb/issues/events{/number}","events_url":"https://api.github.com/repos/arangodb/arangodb/events","assignees_url":"https://api.github.com/repos/arangodb/arangodb/assignees{/user}","branches_url":"https://api.github.com/repos/arangodb/arangodb/branches{/branch}","tags_url":"https://api.github.com/repos/arangodb/arangodb/tags","blobs_url":"https://api.github.com/repos/arangodb/arangodb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/arangodb/arangodb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/arangodb/arangodb/git/refs{/sha}","trees_url":"https://api.github.com/repos/arangodb/arangodb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/arangodb/arangodb/statuses/{sha}","languages_url":"https://api.github.com/repos/arangodb/arangodb/languages","stargazers_url":"https://api.github.com/repos/arangodb/arangodb/stargazers","contributors_url":"https://api.github.com/repos/arangodb/arangodb/contributors","subscribers_url":"https://api.github.com/repos/arangodb/arangodb/subscribers","subscription_url":"https://api.github.com/repos/arangodb/arangodb/subscription","commits_url":"https://api.github.com/repos/arangodb/arangodb/commits{/sha}","git_commits_url":"https://api.github.com/repos/arangodb/arangodb/git/commits{/sha}","comments_url":"https://api.github.com/repos/arangodb/arangodb/comments{/number}","issue_comment_url":"https://api.github.com/repos/arangodb/arangodb/issues/comments{/number}","contents_url":"https://api.github.com/repos/arangodb/arangodb/contents/{+path}","compare_url":"https://api.github.com/repos/arangodb/arangodb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/arangodb/arangodb/merges","archive_url":"https://api.github.com/repos/arangodb/arangodb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/arangodb/arangodb/downloads","issues_url":"https://api.github.com/repos/arangodb/arangodb/issues{/number}","pulls_url":"https://api.github.com/repos/arangodb/arangodb/pulls{/number}","milestones_url":"https://api.github.com/repos/arangodb/arangodb/milestones{/number}","notifications_url":"https://api.github.com/repos/arangodb/arangodb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/arangodb/arangodb/labels{/name}","releases_url":"https://api.github.com/repos/arangodb/arangodb/releases{/id}","deployments_url":"https://api.github.com/repos/arangodb/arangodb/deployments","created_at":"2011-10-26T06:42:00Z","updated_at":"2018-05-15T12:30:03Z","pushed_at":"2018-05-15T12:41:08Z","git_url":"git://github.com/arangodb/arangodb.git","ssh_url":"git@github.com:arangodb/arangodb.git","clone_url":"https://github.com/arangodb/arangodb.git","svn_url":"https://github.com/arangodb/arangodb","homepage":"https://www.arangodb.com","size":822804,"stargazers_count":5651,"watchers_count":5651,"language":"C++","has_issues":true,"has_projects":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":421,"mirror_url":null,"archived":false,"open_issues_count":407,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":421,"open_issues":407,"watchers":5651,"default_branch":"devel"}},"base":{"label":"arangodb:3.3","ref":"3.3","sha":"72420495c14b1366a277a7639f0e1f700d1ea2ac","user":{"login":"arangodb","id":5547849,"avatar_url":"https://avatars0.githubusercontent.com/u/5547849?v=4","gravatar_id":"","url":"https://api.github.com/users/arangodb","html_url":"https://github.com/arangodb","followers_url":"https://api.github.com/users/arangodb/followers","following_url":"https://api.github.com/users/arangodb/following{/other_user}","gists_url":"https://api.github.com/users/arangodb/gists{/gist_id}","starred_url":"https://api.github.com/users/arangodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arangodb/subscriptions","organizations_url":"https://api.github.com/users/arangodb/orgs","repos_url":"https://api.github.com/users/arangodb/repos","events_url":"https://api.github.com/users/arangodb/events{/privacy}","received_events_url":"https://api.github.com/users/arangodb/received_events","type":"Organization","site_admin":false},"repo":{"id":2649214,"name":"arangodb","full_name":"arangodb/arangodb","owner":{"login":"arangodb","id":5547849,"avatar_url":"https://avatars0.githubusercontent.com/u/5547849?v=4","gravatar_id":"","url":"https://api.github.com/users/arangodb","html_url":"https://github.com/arangodb","followers_url":"https://api.github.com/users/arangodb/followers","following_url":"https://api.github.com/users/arangodb/following{/other_user}","gists_url":"https://api.github.com/users/arangodb/gists{/gist_id}","starred_url":"https://api.github.com/users/arangodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arangodb/subscriptions","organizations_url":"https://api.github.com/users/arangodb/orgs","repos_url":"https://api.github.com/users/arangodb/repos","events_url":"https://api.github.com/users/arangodb/events{/privacy}","received_events_url":"https://api.github.com/users/arangodb/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/arangodb/arangodb","description":"🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.","fork":false,"url":"https://api.github.com/repos/arangodb/arangodb","forks_url":"https://api.github.com/repos/arangodb/arangodb/forks","keys_url":"https://api.github.com/repos/arangodb/arangodb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/arangodb/arangodb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/arangodb/arangodb/teams","hooks_url":"https://api.github.com/repos/arangodb/arangodb/hooks","issue_events_url":"https://api.github.com/repos/arangodb/arangodb/issues/events{/number}","events_url":"https://api.github.com/repos/arangodb/arangodb/events","assignees_url":"https://api.github.com/repos/arangodb/arangodb/assignees{/user}","branches_url":"https://api.github.com/repos/arangodb/arangodb/branches{/branch}","tags_url":"https://api.github.com/repos/arangodb/arangodb/tags","blobs_url":"https://api.github.com/repos/arangodb/arangodb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/arangodb/arangodb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/arangodb/arangodb/git/refs{/sha}","trees_url":"https://api.github.com/repos/arangodb/arangodb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/arangodb/arangodb/statuses/{sha}","languages_url":"https://api.github.com/repos/arangodb/arangodb/languages","stargazers_url":"https://api.github.com/repos/arangodb/arangodb/stargazers","contributors_url":"https://api.github.com/repos/arangodb/arangodb/contributors","subscribers_url":"https://api.github.com/repos/arangodb/arangodb/subscribers","subscription_url":"https://api.github.com/repos/arangodb/arangodb/subscription","commits_url":"https://api.github.com/repos/arangodb/arangodb/commits{/sha}","git_commits_url":"https://api.github.com/repos/arangodb/arangodb/git/commits{/sha}","comments_url":"https://api.github.com/repos/arangodb/arangodb/comments{/number}","issue_comment_url":"https://api.github.com/repos/arangodb/arangodb/issues/comments{/number}","contents_url":"https://api.github.com/repos/arangodb/arangodb/contents/{+path}","compare_url":"https://api.github.com/repos/arangodb/arangodb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/arangodb/arangodb/merges","archive_url":"https://api.github.com/repos/arangodb/arangodb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/arangodb/arangodb/downloads","issues_url":"https://api.github.com/repos/arangodb/arangodb/issues{/number}","pulls_url":"https://api.github.com/repos/arangodb/arangodb/pulls{/number}","milestones_url":"https://api.github.com/repos/arangodb/arangodb/milestones{/number}","notifications_url":"https://api.github.com/repos/arangodb/arangodb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/arangodb/arangodb/labels{/name}","releases_url":"https://api.github.com/repos/arangodb/arangodb/releases{/id}","deployments_url":"https://api.github.com/repos/arangodb/arangodb/deployments","created_at":"2011-10-26T06:42:00Z","updated_at":"2018-05-15T12:30:03Z","pushed_at":"2018-05-15T12:41:08Z","git_url":"git://github.com/arangodb/arangodb.git","ssh_url":"git@github.com:arangodb/arangodb.git","clone_url":"https://github.com/arangodb/arangodb.git","svn_url":"https://github.com/arangodb/arangodb","homepage":"https://www.arangodb.com","size":822804,"stargazers_count":5651,"watchers_count":5651,"language":"C++","has_issues":true,"has_projects":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":421,"mirror_url":null,"archived":false,"open_issues_count":407,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":421,"open_issues":407,"watchers":5651,"default_branch":"devel"}},"_links":{"self":{"href":"https://api.github.com/repos/arangodb/arangodb/pulls/5140"},"html":{"href":"https://github.com/arangodb/arangodb/pull/5140"},"issue":{"href":"https://api.github.com/repos/arangodb/arangodb/issues/5140"},"comments":{"href":"https://api.github.com/repos/arangodb/arangodb/issues/5140/comments"},"review_comments":{"href":"https://api.github.com/repos/arangodb/arangodb/pulls/5140/comments"},"review_comment":{"href":"https://api.github.com/repos/arangodb/arangodb/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/arangodb/arangodb/pulls/5140/commits"},"statuses":{"href":"https://api.github.com/repos/arangodb/arangodb/statuses/227af8f4d89739ffe0e084335e823f17ef405207"}},"author_association":"CONTRIBUTOR","merged":true,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":{"login":"jsteemann","id":1423118,"avatar_url":"https://avatars0.githubusercontent.com/u/1423118?v=4","gravatar_id":"","url":"https://api.github.com/users/jsteemann","html_url":"https://github.com/jsteemann","followers_url":"https://api.github.com/users/jsteemann/followers","following_url":"https://api.github.com/users/jsteemann/following{/other_user}","gists_url":"https://api.github.com/users/jsteemann/gists{/gist_id}","starred_url":"https://api.github.com/users/jsteemann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsteemann/subscriptions","organizations_url":"https://api.github.com/users/jsteemann/orgs","repos_url":"https://api.github.com/users/jsteemann/repos","events_url":"https://api.github.com/users/jsteemann/events{/privacy}","received_events_url":"https://api.github.com/users/jsteemann/received_events","type":"User","site_admin":false},"comments":2,"review_comments":0,"maintainer_can_modify":false,"commits":22,"additions":2488,"deletions":786,"changed_files":61}}
{"actor":{"display_login":"jsteemann"}}