{"action":"created","issue":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/4057","repository_url":"https://api.github.com/repos/prometheus/prometheus","labels_url":"https://api.github.com/repos/prometheus/prometheus/issues/4057/labels{/name}","comments_url":"https://api.github.com/repos/prometheus/prometheus/issues/4057/comments","events_url":"https://api.github.com/repos/prometheus/prometheus/issues/4057/events","html_url":"https://github.com/prometheus/prometheus/issues/4057","id":312139777,"node_id":"MDU6SXNzdWUzMTIxMzk3Nzc=","number":4057,"title":"Using `Select()` for both sample+metadata queries is unsuitable for remote storage implementations","user":{"login":"juliusv","id":538008,"node_id":"MDQ6VXNlcjUzODAwOA==","avatar_url":"https://avatars2.githubusercontent.com/u/538008?v=4","gravatar_id":"","url":"https://api.github.com/users/juliusv","html_url":"https://github.com/juliusv","followers_url":"https://api.github.com/users/juliusv/followers","following_url":"https://api.github.com/users/juliusv/following{/other_user}","gists_url":"https://api.github.com/users/juliusv/gists{/gist_id}","starred_url":"https://api.github.com/users/juliusv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/juliusv/subscriptions","organizations_url":"https://api.github.com/users/juliusv/orgs","repos_url":"https://api.github.com/users/juliusv/repos","events_url":"https://api.github.com/users/juliusv/events{/privacy}","received_events_url":"https://api.github.com/users/juliusv/received_events","type":"User","site_admin":false},"labels":[{"id":689154505,"node_id":"MDU6TGFiZWw2ODkxNTQ1MDU=","url":"https://api.github.com/repos/prometheus/prometheus/labels/component/api","name":"component/api","color":"0052cc","default":false},{"id":365567268,"node_id":"MDU6TGFiZWwzNjU1NjcyNjg=","url":"https://api.github.com/repos/prometheus/prometheus/labels/component/local%20storage","name":"component/local storage","color":"0052cc","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-04-06T22:34:50Z","updated_at":"2018-11-09T14:00:22Z","closed_at":null,"author_association":"MEMBER","body":"The `storage.Querier` interface has a `Select()` method which can be used to retrieve either only metadata or bulk sample data of series. In the local storage case, this works nicely, as sample data is loaded lazily when accessed, and thus metadata-only queries never over-fetch sample data. This breaks down for remote storage implementations of the same interface, where we cannot afford to lazily fetch sample data only as it's accessed (that would require more remote round trips). We need some way of knowing beforehand (when we call `Select()`) whether a query is metadata-only or whether it actually needs sample data. Either a separate method or query parameter would work.\r\n\r\nContext: Cortex reuses Prometheus's web API and PromQL packages to offer comparable functionality based on a different storage engine. In the past, we were able to solve this issue because the Prometheus web API packages allowed injecting two different queriers: one that would be used as part of the PromQL engine (the full sample querier) and one that would be used for metadata queries. This got broken in https://github.com/prometheus/prometheus/commit/7ccd4b39b89e936bf0f9c47811e3fe2b20d1fe3b#diff-d81f5cda89ea7b129ba708b586c2bc83L132, due to a PromQL engine restructuring (the querier is not part of the engine anymore, it gets passed in for every query, and the web API only knows about one querier now, because we cannot encapsulate the other one in the engine anymore).\r\n\r\nThe issue about this on the Cortex side is https://github.com/weaveworks/cortex/issues/787"},"comment":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/comments/437367684","html_url":"https://github.com/prometheus/prometheus/issues/4057#issuecomment-437367684","issue_url":"https://api.github.com/repos/prometheus/prometheus/issues/4057","id":437367684,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNzM2NzY4NA==","user":{"login":"krasi-georgiev","id":8903888,"node_id":"MDQ6VXNlcjg5MDM4ODg=","avatar_url":"https://avatars0.githubusercontent.com/u/8903888?v=4","gravatar_id":"","url":"https://api.github.com/users/krasi-georgiev","html_url":"https://github.com/krasi-georgiev","followers_url":"https://api.github.com/users/krasi-georgiev/followers","following_url":"https://api.github.com/users/krasi-georgiev/following{/other_user}","gists_url":"https://api.github.com/users/krasi-georgiev/gists{/gist_id}","starred_url":"https://api.github.com/users/krasi-georgiev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krasi-georgiev/subscriptions","organizations_url":"https://api.github.com/users/krasi-georgiev/orgs","repos_url":"https://api.github.com/users/krasi-georgiev/repos","events_url":"https://api.github.com/users/krasi-georgiev/events{/privacy}","received_events_url":"https://api.github.com/users/krasi-georgiev/received_events","type":"User","site_admin":false},"created_at":"2018-11-09T14:00:22Z","updated_at":"2018-11-09T14:00:22Z","author_association":"MEMBER","body":"@juliusv does anything need to be done on the tsdb side? \r\nIf not would you mind removing the local storage label so that it doesn't appear i my queue :stuck_out_tongue_winking_eye: "}}
{"actor":{"display_login":"krasi-georgiev"}}