{"action":"created","issue":{"url":"https://api.github.com/repos/metabase/metabase/issues/7047","repository_url":"https://api.github.com/repos/metabase/metabase","labels_url":"https://api.github.com/repos/metabase/metabase/issues/7047/labels{/name}","comments_url":"https://api.github.com/repos/metabase/metabase/issues/7047/comments","events_url":"https://api.github.com/repos/metabase/metabase/issues/7047/events","html_url":"https://github.com/metabase/metabase/pull/7047","id":301492500,"number":7047,"title":"HTTP/REST driver [PoC]","user":{"login":"tlrobinson","id":18193,"avatar_url":"https://avatars0.githubusercontent.com/u/18193?v=4","gravatar_id":"","url":"https://api.github.com/users/tlrobinson","html_url":"https://github.com/tlrobinson","followers_url":"https://api.github.com/users/tlrobinson/followers","following_url":"https://api.github.com/users/tlrobinson/following{/other_user}","gists_url":"https://api.github.com/users/tlrobinson/gists{/gist_id}","starred_url":"https://api.github.com/users/tlrobinson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tlrobinson/subscriptions","organizations_url":"https://api.github.com/users/tlrobinson/orgs","repos_url":"https://api.github.com/users/tlrobinson/repos","events_url":"https://api.github.com/users/tlrobinson/events{/privacy}","received_events_url":"https://api.github.com/users/tlrobinson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-03-01T17:30:11Z","updated_at":"2018-03-05T19:06:53Z","closed_at":null,"author_association":"OWNER","pull_request":{"url":"https://api.github.com/repos/metabase/metabase/pulls/7047","html_url":"https://github.com/metabase/metabase/pull/7047","diff_url":"https://github.com/metabase/metabase/pull/7047.diff","patch_url":"https://github.com/metabase/metabase/pull/7047.patch"},"body":"This is a prototype of a generic HTTP/REST driver, i.e. #4831\r\n\r\nSo far it's pretty basic. The \"native\" query is a JSON object that must have a `url` property:\r\n\r\n```json\r\n{\"url\":\"https://api.coinmarketcap.com/v1/ticker/\"}\r\n```\r\n\r\nBy default it assumes the result is an array and it uses the properties in the first item as the columns:\r\n\r\n<img width=\"1035\" alt=\"screenshot 2018-03-02 00 15 20\" src=\"https://user-images.githubusercontent.com/18193/36858788-d51ad1c4-1dae-11e8-92ee-4563ddd4b120.png\">\r\n\r\nYou can also specify the fields you want (this filters in the backend, so you still have to fetch the full results):\r\n\r\n```json\r\n{\r\n  \"url\":\"https://api.coinmarketcap.com/v1/ticker/\",\r\n  \"result\": {\r\n    \"fields\": [\"id\", \"price_usd\"]\r\n  }\r\n}\r\n```\r\n\r\nAnd if the result isn't an array you can specify the path to the array:\r\n\r\n```json\r\n{\r\n  \"url\":\"https://blockchain.info/blocks?format=json\",\r\n  \"result\":{\r\n    \"path\":\"blocks\",\r\n    \"fields\":[\"height\",\"time\"]\r\n  }\r\n}\r\n```\r\n\r\nBoth the `path` and `fields` are actually [JSONPath](https://github.com/json-path/JsonPath).\r\n\r\nIn addition to \"native\" query you can pre-define \"tables\" that will appear in the query builder using a JSON definition in the database configuration that looks like this:\r\n\r\n```json\r\n{\r\n  \"tables\": [\r\n    {\r\n      \"name\": \"Blocks\",\r\n      \"url\": \"https://blockchain.info/blocks?format=json\",\r\n      \"result\": { \"path\": \"blocks\" }\r\n    },\r\n    {\r\n      \"name\": \"Prices\",\r\n      \"url\": \"https://api.coinmarketcap.com/v1/ticker/\"\r\n    }\r\n  ]\r\n}\r\n```\r\n\r\n<img width=\"997\" alt=\"screenshot 2018-03-02 00 31 14\" src=\"https://user-images.githubusercontent.com/18193/36859482-10201c0a-1db1-11e8-9add-cbcc84806183.png\">\r\n\r\nEventually you'd be able to define fields that you can filter on (map fields to variables in a URL template string and/or query string arguments), and specify more rich metadata about the columns, etc.\r\n\r\nOther open questions include:\r\n\r\n* authentication\r\n* pagination\r\n* aggregation/grouping?"},"comment":{"url":"https://api.github.com/repos/metabase/metabase/issues/comments/370527883","html_url":"https://github.com/metabase/metabase/pull/7047#issuecomment-370527883","issue_url":"https://api.github.com/repos/metabase/metabase/issues/7047","id":370527883,"user":{"login":"jornh","id":766423,"avatar_url":"https://avatars3.githubusercontent.com/u/766423?v=4","gravatar_id":"","url":"https://api.github.com/users/jornh","html_url":"https://github.com/jornh","followers_url":"https://api.github.com/users/jornh/followers","following_url":"https://api.github.com/users/jornh/following{/other_user}","gists_url":"https://api.github.com/users/jornh/gists{/gist_id}","starred_url":"https://api.github.com/users/jornh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jornh/subscriptions","organizations_url":"https://api.github.com/users/jornh/orgs","repos_url":"https://api.github.com/users/jornh/repos","events_url":"https://api.github.com/users/jornh/events{/privacy}","received_events_url":"https://api.github.com/users/jornh/received_events","type":"User","site_admin":false},"created_at":"2018-03-05T19:06:53Z","updated_at":"2018-03-05T19:06:53Z","author_association":"CONTRIBUTOR","body":"## First something to wet the appetite for others to try this out\r\n\r\nI kicked the tires on this a bit over the weekend. Awesome stuff @tlrobinson! At first I also stumbled into `Fix defn ordering`  https://github.com/metabase/metabase/pull/7047/commits/9dd963b7de200a783f2cfcf6bd5371b9cf371e8e that you already fixed above. I then replicated your examples above, then tried: \r\n```\r\n{\"url\": \"https://api.github.com/users/metabase/repos\"}\r\n``` \r\nwhich resulted in:\r\n![classic dash dashboard metabase - google chrome](https://user-images.githubusercontent.com/766423/36993320-d13c2c7c-20ad-11e8-8519-36a00790bf53.jpg)\r\n\r\nTo have a go, here's a [gist](https://gist.github.com/jornh/a8e7262830c809214bc6b79f755c3530) with a few quick steps to build and run. You would of course need to checkout the `http-driver` branch, not `release-0.28.2`.\r\n\r\n## Low `auth` fruit with authentication headers?\r\n\r\nI then tried connection to a few resources at logentries.com. They require a `x-api-key: XXXX` header and I guess that would also apply to a lot of other APIs. So maybe that's low hanging 🍏? I managed to hack/hardcode that into `http.clj`. To avoid hard coding would something like below added to the \"native queries\" be too far fetched to begin with? \r\n\r\n```\r\n{\r\n  ...\r\n  \"headers\":{\r\n    \"x-api-key\":\"XXXXXXXX-XXXX-XXXX\",\r\n    \"another-random-header\":[\"multiple\",\"values\"]\r\n  }\r\n}\r\n```"}}
{"actor":{"display_login":"jornh"}}