{"action":"created","issue":{"url":"https://api.github.com/repos/citusdata/citus/issues/414","repository_url":"https://api.github.com/repos/citusdata/citus","labels_url":"https://api.github.com/repos/citusdata/citus/issues/414/labels{/name}","comments_url":"https://api.github.com/repos/citusdata/citus/issues/414/comments","events_url":"https://api.github.com/repos/citusdata/citus/issues/414/events","html_url":"https://github.com/citusdata/citus/pull/414","id":144229081,"number":414,"title":"Add EXPLAIN for simple distributed queries","user":{"login":"marcocitus","id":7127240,"avatar_url":"https://avatars.githubusercontent.com/u/7127240?v=3","gravatar_id":"","url":"https://api.github.com/users/marcocitus","html_url":"https://github.com/marcocitus","followers_url":"https://api.github.com/users/marcocitus/followers","following_url":"https://api.github.com/users/marcocitus/following{/other_user}","gists_url":"https://api.github.com/users/marcocitus/gists{/gist_id}","starred_url":"https://api.github.com/users/marcocitus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marcocitus/subscriptions","organizations_url":"https://api.github.com/users/marcocitus/orgs","repos_url":"https://api.github.com/users/marcocitus/repos","events_url":"https://api.github.com/users/marcocitus/events{/privacy}","received_events_url":"https://api.github.com/users/marcocitus/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/citusdata/citus/labels/needs%20review","name":"needs review","color":"eeeeee"}],"state":"open","locked":false,"assignee":{"login":"anarazel","id":1598211,"avatar_url":"https://avatars.githubusercontent.com/u/1598211?v=3","gravatar_id":"","url":"https://api.github.com/users/anarazel","html_url":"https://github.com/anarazel","followers_url":"https://api.github.com/users/anarazel/followers","following_url":"https://api.github.com/users/anarazel/following{/other_user}","gists_url":"https://api.github.com/users/anarazel/gists{/gist_id}","starred_url":"https://api.github.com/users/anarazel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anarazel/subscriptions","organizations_url":"https://api.github.com/users/anarazel/orgs","repos_url":"https://api.github.com/users/anarazel/repos","events_url":"https://api.github.com/users/anarazel/events{/privacy}","received_events_url":"https://api.github.com/users/anarazel/received_events","type":"User","site_admin":false},"milestone":null,"comments":4,"created_at":"2016-03-29T11:02:29Z","updated_at":"2016-04-18T03:54:32Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/citusdata/citus/pulls/414","html_url":"https://github.com/citusdata/citus/pull/414","diff_url":"https://github.com/citusdata/citus/pull/414.diff","patch_url":"https://github.com/citusdata/citus/pull/414.patch"},"body":"This change adds EXPLAIN for simple distributed queries, meaning queries that do not contain re-partition jobs. The EXPLAIN output consists of two parts: The distributed query and the master query. \r\n\r\nThe distributed query part shows the remote EXPLAIN plan for either one arbitrary task, or all tasks if citus.explain_all_tasks is on. A few limitations to note:\r\n- A remote EXPLAIN may error out when explaining a broadcast join while the shards for the small table have not yet been fetched. An error message is displayed advising to run the query first.\r\n- When citus.explain_all_tasks is on, EXPLAIN plans are retrieved sequentially, which may take a long time for EXPLAIN ANALYZE. \r\n- The remote EXPLAIN is not shown by auto_explain, because auto_explain doesn't call the explain hook. We'd need to implement our own auto_explain logic to support it.\r\n\r\nThe master query part uses the regular EXPLAIN logic to show the plan of the master query by calling ExplainOnePlan. \r\n- During EXPLAIN ANALYZE, some tasks will be executed twice, once while fetching the remote EXPLAIN ANALYZE step and again during distributed execution.\r\n- The execution time in EXPLAIN ANALYZE shows the execution time of the master query, which seemed more useful to understand performance than the overall execution time.\r\n\r\nThere are a few quirks as a result of limitations in the Explain functions. For example, in the JSON format, the Plan for a Task is represented as an array in an array. The outer array is unnecessary, but there is no nice way to avoid it.\r\n\r\nA number of existing regression tests used EXPLAIN and relied on the error message saying that EXPLAIN was not supported. I first tried adding the EXPLAIN output to the expected output. However, some tests use queries that are not supported by the current implementation EXPLAIN and I didn't want to move the problem forward. I then tried changing to SELECT or CREATE TEMPORARY TABLE AS, but some tests use EXPLAIN solely to test the planner logic, while their execution actually fails. In the end, I added a citus.explain_distributed_queries option that is on by default, but can be turned off for regression tests and produces an error message as before. I did change the message for correctness, so there are a lot of changes in the regression tests. I can't think of a use-case where you might want to disable EXPLAIN, other than regression tests.\r\n\r\nCloses #232 \r\nCloses #15 "},"comment":{"url":"https://api.github.com/repos/citusdata/citus/issues/comments/211182243","html_url":"https://github.com/citusdata/citus/pull/414#issuecomment-211182243","issue_url":"https://api.github.com/repos/citusdata/citus/issues/414","id":211182243,"user":{"login":"anarazel","id":1598211,"avatar_url":"https://avatars.githubusercontent.com/u/1598211?v=3","gravatar_id":"","url":"https://api.github.com/users/anarazel","html_url":"https://github.com/anarazel","followers_url":"https://api.github.com/users/anarazel/followers","following_url":"https://api.github.com/users/anarazel/following{/other_user}","gists_url":"https://api.github.com/users/anarazel/gists{/gist_id}","starred_url":"https://api.github.com/users/anarazel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anarazel/subscriptions","organizations_url":"https://api.github.com/users/anarazel/orgs","repos_url":"https://api.github.com/users/anarazel/repos","events_url":"https://api.github.com/users/anarazel/events{/privacy}","received_events_url":"https://api.github.com/users/anarazel/received_events","type":"User","site_admin":false},"created_at":"2016-04-18T03:54:32Z","updated_at":"2016-04-18T03:54:32Z","body":"@marcocitus There's actually pending comments to be addressed and a rebase."}}