{"action":"created","comment":{"url":"https://api.github.com/repos/metabase/metabase/pulls/comments/62124266","id":62124266,"diff_hunk":"@@ -39,26 +40,25 @@\n \n ;; # NATIVE QUERY PROCESSOR\n \n-(defn- eval-raw-command\n-  \"Evaluate raw MongoDB javascript code. This must be ran insided the body of a `with-mongo-connection`.\n-\n-     (with-mongo-connection [_ \\\"mongodb://localhost/test\\\"]\n-       (eval-raw-command \\\"db.zips.findOne()\\\"))\n-     -> {\\\"_id\\\" \\\"01001\\\", \\\"city\\\" \\\"AGAWAM\\\", ...}\"\n-  [^String command]\n-  (assert *mongo-connection* \"eval-raw-command must be ran inside the body of with-mongo-connection.\")\n-  (let [^CommandResult result (.doEval ^DB *mongo-connection* command nil)]\n-      (when-not (.ok result)\n-        (throw (.getException result)))\n-      (let [{result \"retval\"} (PersistentArrayMap/create (.toMap result))]\n-        result)))\n-\n (defn process-and-run-native\n   \"Process and run a native MongoDB query.\"\n-  [_ query]\n-  (let [results (eval-raw-command (:query (:native query)))]\n-    (if (sequential? results) results\n-        [results])))\n+  [{{query :query} :native, database :database, :as outer-query}]\n+  {:pre [query]}\n+  (let [query   (u/prog1 (if (string? query)\n+                           (json/parse-string query keyword)\n+                           query)\n+                  (println \"QUERY ->\" <>))\n+        results  (mc/aggregate *mongo-connection* \"users\" query\n+                               :allow-disk-use true)]\n+    ;; As with Druid, we want to use `annotate` on the results of native queries.\n+    ;; Because the Generic SQL driver was written in ancient times, it has its own internal implementation of `annotate`\n+    ;; (which preserves the order of columns in the results) and as a result the `annotate` middleware isn't applied for `:native` queries.\n+    ;; Thus we need to manually run it ourself here.\n+    ;; TODO - it would be nice if we could let the middleware take care of `annotate` for us. This will likely require a new `IDriver` method\n+    ;; returns whether or not `annotate` should be done to native query results.\n+    (annotate/annotate outer-query (if (sequential? results)","path":"src/metabase/driver/mongo/query_processor.clj","position":46,"original_position":46,"commit_id":"3187a89b757f7a5c0c0d135124e7c37ee0146eae","original_commit_id":"3187a89b757f7a5c0c0d135124e7c37ee0146eae","user":{"login":"agilliland","id":1987787,"avatar_url":"https://avatars.githubusercontent.com/u/1987787?v=3","gravatar_id":"","url":"https://api.github.com/users/agilliland","html_url":"https://github.com/agilliland","followers_url":"https://api.github.com/users/agilliland/followers","following_url":"https://api.github.com/users/agilliland/following{/other_user}","gists_url":"https://api.github.com/users/agilliland/gists{/gist_id}","starred_url":"https://api.github.com/users/agilliland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agilliland/subscriptions","organizations_url":"https://api.github.com/users/agilliland/orgs","repos_url":"https://api.github.com/users/agilliland/repos","events_url":"https://api.github.com/users/agilliland/events{/privacy}","received_events_url":"https://api.github.com/users/agilliland/received_events","type":"User","site_admin":false},"body":"Given that the output structures between `process-native` and `process-structured` are clearly quite different, it seems like this is something that we could handle pretty easily.  In fact, since one is a map and the other is a collection then couldn't we just do something like this in the QP middleware function ...\r\n\r\n```\r\n(if-not (map? query-result)\r\n (annotate/annotate query-result)\r\n query-result)\r\n```","created_at":"2016-05-04T22:15:57Z","updated_at":"2016-05-04T22:15:57Z","html_url":"https://github.com/metabase/metabase/pull/2534#discussion_r62124266","pull_request_url":"https://api.github.com/repos/metabase/metabase/pulls/2534","_links":{"self":{"href":"https://api.github.com/repos/metabase/metabase/pulls/comments/62124266"},"html":{"href":"https://github.com/metabase/metabase/pull/2534#discussion_r62124266"},"pull_request":{"href":"https://api.github.com/repos/metabase/metabase/pulls/2534"}}},"pull_request":{"url":"https://api.github.com/repos/metabase/metabase/pulls/2534","id":68937463,"html_url":"https://github.com/metabase/metabase/pull/2534","diff_url":"https://github.com/metabase/metabase/pull/2534.diff","patch_url":"https://github.com/metabase/metabase/pull/2534.patch","issue_url":"https://api.github.com/repos/metabase/metabase/issues/2534","number":2534,"state":"open","locked":false,"title":"Allow native queries for MongoDB :scream_cat:","user":{"login":"camsaul","id":1455846,"avatar_url":"https://avatars.githubusercontent.com/u/1455846?v=3","gravatar_id":"","url":"https://api.github.com/users/camsaul","html_url":"https://github.com/camsaul","followers_url":"https://api.github.com/users/camsaul/followers","following_url":"https://api.github.com/users/camsaul/following{/other_user}","gists_url":"https://api.github.com/users/camsaul/gists{/gist_id}","starred_url":"https://api.github.com/users/camsaul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/camsaul/subscriptions","organizations_url":"https://api.github.com/users/camsaul/orgs","repos_url":"https://api.github.com/users/camsaul/repos","events_url":"https://api.github.com/users/camsaul/events{/privacy}","received_events_url":"https://api.github.com/users/camsaul/received_events","type":"User","site_admin":false},"body":"Since everybody polled prefered the Mongo Aggregation Pipeline as the native option here it is.\r\n\r\nImplements #2072.","created_at":"2016-05-04T21:46:50Z","updated_at":"2016-05-04T22:15:57Z","closed_at":null,"merged_at":null,"merge_commit_sha":"877e93659d8df66c7a3b3f6800530a7c104be35b","assignee":{"login":"camsaul","id":1455846,"avatar_url":"https://avatars.githubusercontent.com/u/1455846?v=3","gravatar_id":"","url":"https://api.github.com/users/camsaul","html_url":"https://github.com/camsaul","followers_url":"https://api.github.com/users/camsaul/followers","following_url":"https://api.github.com/users/camsaul/following{/other_user}","gists_url":"https://api.github.com/users/camsaul/gists{/gist_id}","starred_url":"https://api.github.com/users/camsaul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/camsaul/subscriptions","organizations_url":"https://api.github.com/users/camsaul/orgs","repos_url":"https://api.github.com/users/camsaul/repos","events_url":"https://api.github.com/users/camsaul/events{/privacy}","received_events_url":"https://api.github.com/users/camsaul/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/metabase/metabase/milestones/18","html_url":"https://github.com/metabase/metabase/milestones/0.18.0","labels_url":"https://api.github.com/repos/metabase/metabase/milestones/18/labels","id":1736953,"number":18,"title":"0.18.0","description":"Code Complete: May TBD","creator":{"login":"salsakran","id":107439,"avatar_url":"https://avatars.githubusercontent.com/u/107439?v=3","gravatar_id":"","url":"https://api.github.com/users/salsakran","html_url":"https://github.com/salsakran","followers_url":"https://api.github.com/users/salsakran/followers","following_url":"https://api.github.com/users/salsakran/following{/other_user}","gists_url":"https://api.github.com/users/salsakran/gists{/gist_id}","starred_url":"https://api.github.com/users/salsakran/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/salsakran/subscriptions","organizations_url":"https://api.github.com/users/salsakran/orgs","repos_url":"https://api.github.com/users/salsakran/repos","events_url":"https://api.github.com/users/salsakran/events{/privacy}","received_events_url":"https://api.github.com/users/salsakran/received_events","type":"User","site_admin":false},"open_issues":16,"closed_issues":6,"state":"open","created_at":"2016-04-28T17:19:48Z","updated_at":"2016-05-04T21:56:35Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/metabase/metabase/pulls/2534/commits","review_comments_url":"https://api.github.com/repos/metabase/metabase/pulls/2534/comments","review_comment_url":"https://api.github.com/repos/metabase/metabase/pulls/comments{/number}","comments_url":"https://api.github.com/repos/metabase/metabase/issues/2534/comments","statuses_url":"https://api.github.com/repos/metabase/metabase/statuses/3187a89b757f7a5c0c0d135124e7c37ee0146eae","head":{"label":"metabase:mongo-native-queries","ref":"mongo-native-queries","sha":"3187a89b757f7a5c0c0d135124e7c37ee0146eae","user":{"login":"metabase","id":10520629,"avatar_url":"https://avatars.githubusercontent.com/u/10520629?v=3","gravatar_id":"","url":"https://api.github.com/users/metabase","html_url":"https://github.com/metabase","followers_url":"https://api.github.com/users/metabase/followers","following_url":"https://api.github.com/users/metabase/following{/other_user}","gists_url":"https://api.github.com/users/metabase/gists{/gist_id}","starred_url":"https://api.github.com/users/metabase/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/metabase/subscriptions","organizations_url":"https://api.github.com/users/metabase/orgs","repos_url":"https://api.github.com/users/metabase/repos","events_url":"https://api.github.com/users/metabase/events{/privacy}","received_events_url":"https://api.github.com/users/metabase/received_events","type":"Organization","site_admin":false},"repo":{"id":30203935,"name":"metabase","full_name":"metabase/metabase","owner":{"login":"metabase","id":10520629,"avatar_url":"https://avatars.githubusercontent.com/u/10520629?v=3","gravatar_id":"","url":"https://api.github.com/users/metabase","html_url":"https://github.com/metabase","followers_url":"https://api.github.com/users/metabase/followers","following_url":"https://api.github.com/users/metabase/following{/other_user}","gists_url":"https://api.github.com/users/metabase/gists{/gist_id}","starred_url":"https://api.github.com/users/metabase/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/metabase/subscriptions","organizations_url":"https://api.github.com/users/metabase/orgs","repos_url":"https://api.github.com/users/metabase/repos","events_url":"https://api.github.com/users/metabase/events{/privacy}","received_events_url":"https://api.github.com/users/metabase/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/metabase/metabase","description":"The simplest, fastest way to get business intelligence and analytics  to everyone in your company :yum:","fork":false,"url":"https://api.github.com/repos/metabase/metabase","forks_url":"https://api.github.com/repos/metabase/metabase/forks","keys_url":"https://api.github.com/repos/metabase/metabase/keys{/key_id}","collaborators_url":"https://api.github.com/repos/metabase/metabase/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/metabase/metabase/teams","hooks_url":"https://api.github.com/repos/metabase/metabase/hooks","issue_events_url":"https://api.github.com/repos/metabase/metabase/issues/events{/number}","events_url":"https://api.github.com/repos/metabase/metabase/events","assignees_url":"https://api.github.com/repos/metabase/metabase/assignees{/user}","branches_url":"https://api.github.com/repos/metabase/metabase/branches{/branch}","tags_url":"https://api.github.com/repos/metabase/metabase/tags","blobs_url":"https://api.github.com/repos/metabase/metabase/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/metabase/metabase/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/metabase/metabase/git/refs{/sha}","trees_url":"https://api.github.com/repos/metabase/metabase/git/trees{/sha}","statuses_url":"https://api.github.com/repos/metabase/metabase/statuses/{sha}","languages_url":"https://api.github.com/repos/metabase/metabase/languages","stargazers_url":"https://api.github.com/repos/metabase/metabase/stargazers","contributors_url":"https://api.github.com/repos/metabase/metabase/contributors","subscribers_url":"https://api.github.com/repos/metabase/metabase/subscribers","subscription_url":"https://api.github.com/repos/metabase/metabase/subscription","commits_url":"https://api.github.com/repos/metabase/metabase/commits{/sha}","git_commits_url":"https://api.github.com/repos/metabase/metabase/git/commits{/sha}","comments_url":"https://api.github.com/repos/metabase/metabase/comments{/number}","issue_comment_url":"https://api.github.com/repos/metabase/metabase/issues/comments{/number}","contents_url":"https://api.github.com/repos/metabase/metabase/contents/{+path}","compare_url":"https://api.github.com/repos/metabase/metabase/compare/{base}...{head}","merges_url":"https://api.github.com/repos/metabase/metabase/merges","archive_url":"https://api.github.com/repos/metabase/metabase/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/metabase/metabase/downloads","issues_url":"https://api.github.com/repos/metabase/metabase/issues{/number}","pulls_url":"https://api.github.com/repos/metabase/metabase/pulls{/number}","milestones_url":"https://api.github.com/repos/metabase/metabase/milestones{/number}","notifications_url":"https://api.github.com/repos/metabase/metabase/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/metabase/metabase/labels{/name}","releases_url":"https://api.github.com/repos/metabase/metabase/releases{/id}","deployments_url":"https://api.github.com/repos/metabase/metabase/deployments","created_at":"2015-02-02T19:25:47Z","updated_at":"2016-05-04T09:21:14Z","pushed_at":"2016-05-04T21:56:36Z","git_url":"git://github.com/metabase/metabase.git","ssh_url":"git@github.com:metabase/metabase.git","clone_url":"https://github.com/metabase/metabase.git","svn_url":"https://github.com/metabase/metabase","homepage":"http://metabase.com","size":36766,"stargazers_count":2590,"watchers_count":2590,"language":"Clojure","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":145,"mirror_url":null,"open_issues_count":323,"forks":145,"open_issues":323,"watchers":2590,"default_branch":"master"}},"base":{"label":"metabase:master","ref":"master","sha":"08460da7dc6018a2bf2715bde87c5e9241357203","user":{"login":"metabase","id":10520629,"avatar_url":"https://avatars.githubusercontent.com/u/10520629?v=3","gravatar_id":"","url":"https://api.github.com/users/metabase","html_url":"https://github.com/metabase","followers_url":"https://api.github.com/users/metabase/followers","following_url":"https://api.github.com/users/metabase/following{/other_user}","gists_url":"https://api.github.com/users/metabase/gists{/gist_id}","starred_url":"https://api.github.com/users/metabase/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/metabase/subscriptions","organizations_url":"https://api.github.com/users/metabase/orgs","repos_url":"https://api.github.com/users/metabase/repos","events_url":"https://api.github.com/users/metabase/events{/privacy}","received_events_url":"https://api.github.com/users/metabase/received_events","type":"Organization","site_admin":false},"repo":{"id":30203935,"name":"metabase","full_name":"metabase/metabase","owner":{"login":"metabase","id":10520629,"avatar_url":"https://avatars.githubusercontent.com/u/10520629?v=3","gravatar_id":"","url":"https://api.github.com/users/metabase","html_url":"https://github.com/metabase","followers_url":"https://api.github.com/users/metabase/followers","following_url":"https://api.github.com/users/metabase/following{/other_user}","gists_url":"https://api.github.com/users/metabase/gists{/gist_id}","starred_url":"https://api.github.com/users/metabase/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/metabase/subscriptions","organizations_url":"https://api.github.com/users/metabase/orgs","repos_url":"https://api.github.com/users/metabase/repos","events_url":"https://api.github.com/users/metabase/events{/privacy}","received_events_url":"https://api.github.com/users/metabase/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/metabase/metabase","description":"The simplest, fastest way to get business intelligence and analytics  to everyone in your company :yum:","fork":false,"url":"https://api.github.com/repos/metabase/metabase","forks_url":"https://api.github.com/repos/metabase/metabase/forks","keys_url":"https://api.github.com/repos/metabase/metabase/keys{/key_id}","collaborators_url":"https://api.github.com/repos/metabase/metabase/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/metabase/metabase/teams","hooks_url":"https://api.github.com/repos/metabase/metabase/hooks","issue_events_url":"https://api.github.com/repos/metabase/metabase/issues/events{/number}","events_url":"https://api.github.com/repos/metabase/metabase/events","assignees_url":"https://api.github.com/repos/metabase/metabase/assignees{/user}","branches_url":"https://api.github.com/repos/metabase/metabase/branches{/branch}","tags_url":"https://api.github.com/repos/metabase/metabase/tags","blobs_url":"https://api.github.com/repos/metabase/metabase/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/metabase/metabase/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/metabase/metabase/git/refs{/sha}","trees_url":"https://api.github.com/repos/metabase/metabase/git/trees{/sha}","statuses_url":"https://api.github.com/repos/metabase/metabase/statuses/{sha}","languages_url":"https://api.github.com/repos/metabase/metabase/languages","stargazers_url":"https://api.github.com/repos/metabase/metabase/stargazers","contributors_url":"https://api.github.com/repos/metabase/metabase/contributors","subscribers_url":"https://api.github.com/repos/metabase/metabase/subscribers","subscription_url":"https://api.github.com/repos/metabase/metabase/subscription","commits_url":"https://api.github.com/repos/metabase/metabase/commits{/sha}","git_commits_url":"https://api.github.com/repos/metabase/metabase/git/commits{/sha}","comments_url":"https://api.github.com/repos/metabase/metabase/comments{/number}","issue_comment_url":"https://api.github.com/repos/metabase/metabase/issues/comments{/number}","contents_url":"https://api.github.com/repos/metabase/metabase/contents/{+path}","compare_url":"https://api.github.com/repos/metabase/metabase/compare/{base}...{head}","merges_url":"https://api.github.com/repos/metabase/metabase/merges","archive_url":"https://api.github.com/repos/metabase/metabase/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/metabase/metabase/downloads","issues_url":"https://api.github.com/repos/metabase/metabase/issues{/number}","pulls_url":"https://api.github.com/repos/metabase/metabase/pulls{/number}","milestones_url":"https://api.github.com/repos/metabase/metabase/milestones{/number}","notifications_url":"https://api.github.com/repos/metabase/metabase/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/metabase/metabase/labels{/name}","releases_url":"https://api.github.com/repos/metabase/metabase/releases{/id}","deployments_url":"https://api.github.com/repos/metabase/metabase/deployments","created_at":"2015-02-02T19:25:47Z","updated_at":"2016-05-04T09:21:14Z","pushed_at":"2016-05-04T21:56:36Z","git_url":"git://github.com/metabase/metabase.git","ssh_url":"git@github.com:metabase/metabase.git","clone_url":"https://github.com/metabase/metabase.git","svn_url":"https://github.com/metabase/metabase","homepage":"http://metabase.com","size":36766,"stargazers_count":2590,"watchers_count":2590,"language":"Clojure","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":145,"mirror_url":null,"open_issues_count":323,"forks":145,"open_issues":323,"watchers":2590,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/metabase/metabase/pulls/2534"},"html":{"href":"https://github.com/metabase/metabase/pull/2534"},"issue":{"href":"https://api.github.com/repos/metabase/metabase/issues/2534"},"comments":{"href":"https://api.github.com/repos/metabase/metabase/issues/2534/comments"},"review_comments":{"href":"https://api.github.com/repos/metabase/metabase/pulls/2534/comments"},"review_comment":{"href":"https://api.github.com/repos/metabase/metabase/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/metabase/metabase/pulls/2534/commits"},"statuses":{"href":"https://api.github.com/repos/metabase/metabase/statuses/3187a89b757f7a5c0c0d135124e7c37ee0146eae"}}}}