{"action":"created","issue":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/4184","repository_url":"https://api.github.com/repos/prometheus/prometheus","labels_url":"https://api.github.com/repos/prometheus/prometheus/issues/4184/labels{/name}","comments_url":"https://api.github.com/repos/prometheus/prometheus/issues/4184/comments","events_url":"https://api.github.com/repos/prometheus/prometheus/issues/4184/events","html_url":"https://github.com/prometheus/prometheus/issues/4184","id":325646251,"node_id":"MDU6SXNzdWUzMjU2NDYyNTE=","number":4184,"title":"Determine whether remote/local data overlap should be smaller with `read_recent: false`","user":{"login":"juliusv","id":538008,"node_id":"MDQ6VXNlcjUzODAwOA==","avatar_url":"https://avatars2.githubusercontent.com/u/538008?v=4","gravatar_id":"","url":"https://api.github.com/users/juliusv","html_url":"https://github.com/juliusv","followers_url":"https://api.github.com/users/juliusv/followers","following_url":"https://api.github.com/users/juliusv/following{/other_user}","gists_url":"https://api.github.com/users/juliusv/gists{/gist_id}","starred_url":"https://api.github.com/users/juliusv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/juliusv/subscriptions","organizations_url":"https://api.github.com/users/juliusv/orgs","repos_url":"https://api.github.com/users/juliusv/repos","events_url":"https://api.github.com/users/juliusv/events{/privacy}","received_events_url":"https://api.github.com/users/juliusv/received_events","type":"User","site_admin":false},"labels":[{"id":365567165,"node_id":"MDU6TGFiZWwzNjU1NjcxNjU=","url":"https://api.github.com/repos/prometheus/prometheus/labels/component/remote%20storage","name":"component/remote storage","color":"0052cc","default":false},{"id":365563592,"node_id":"MDU6TGFiZWwzNjU1NjM1OTI=","url":"https://api.github.com/repos/prometheus/prometheus/labels/kind/enhancement","name":"kind/enhancement","color":"c7def8","default":false},{"id":647013637,"node_id":"MDU6TGFiZWw2NDcwMTM2Mzc=","url":"https://api.github.com/repos/prometheus/prometheus/labels/priority/P3","name":"priority/P3","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2018-05-23T10:33:52Z","updated_at":"2018-06-19T19:53:12Z","closed_at":null,"author_association":"MEMBER","body":"On current master, remote read seems to read all points (even recent and without `read_recent: true`) from local+remote storage and also doesn't merge/dedupe them properly. I simply followed https://www.robustperception.io/using-the-cratedb-prometheus-adapter/ and when I execute `prometheus_tsdb_head_samples_appended_total[1m]` as an instant query in Prometheus, I'm getting the following samples (note the duplication):\r\n\r\n```\r\n18221 @1527071278.548\r\n18221 @1527071278.548\r\n18740 @1527071283.548\r\n18740 @1527071283.548\r\n19259 @1527071284.136\r\n19259 @1527071284.136\r\n19778 @1527071289.136\r\n19778 @1527071289.136\r\n20297 @1527071294.136\r\n20297 @1527071294.136\r\n20816 @1527071299.136\r\n20816 @1527071299.136\r\n21335 @1527071304.136\r\n21335 @1527071304.136\r\n21854 @1527071309.136\r\n21854 @1527071309.136\r\n22373 @1527071314.136\r\n22373 @1527071314.136\r\n22892 @1527071319.136\r\n22892 @1527071319.136\r\n23411 @1527071324.136\r\n23411 @1527071324.136\r\n23930 @1527071329.136\r\n23930 @1527071329.136\r\n24449 @1527071334.136\r\n24449 @1527071334.136\r\n```\r\n\r\nAs a result of the duplicate samples, `irate()` shows no output (since it cannot compute a rate between identical samples).\r\n\r\nThe expected behavior would be to only query the remote read endpoint for samples that are older than what is in the local TSDB and *if* we query the remote endpoint, we should dedupe things properly."},"comment":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/comments/398524076","html_url":"https://github.com/prometheus/prometheus/issues/4184#issuecomment-398524076","issue_url":"https://api.github.com/repos/prometheus/prometheus/issues/4184","id":398524076,"node_id":"MDEyOklzc3VlQ29tbWVudDM5ODUyNDA3Ng==","user":{"login":"juliusv","id":538008,"node_id":"MDQ6VXNlcjUzODAwOA==","avatar_url":"https://avatars2.githubusercontent.com/u/538008?v=4","gravatar_id":"","url":"https://api.github.com/users/juliusv","html_url":"https://github.com/juliusv","followers_url":"https://api.github.com/users/juliusv/followers","following_url":"https://api.github.com/users/juliusv/following{/other_user}","gists_url":"https://api.github.com/users/juliusv/gists{/gist_id}","starred_url":"https://api.github.com/users/juliusv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/juliusv/subscriptions","organizations_url":"https://api.github.com/users/juliusv/orgs","repos_url":"https://api.github.com/users/juliusv/repos","events_url":"https://api.github.com/users/juliusv/events{/privacy}","received_events_url":"https://api.github.com/users/juliusv/received_events","type":"User","site_admin":false},"created_at":"2018-06-19T19:53:11Z","updated_at":"2018-06-19T19:53:11Z","author_association":"MEMBER","body":"Changed the title to reflect this."}}
{"actor":{"display_login":"juliusv"}}