{"action":"created","issue":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/1965","repository_url":"https://api.github.com/repos/prometheus/prometheus","labels_url":"https://api.github.com/repos/prometheus/prometheus/issues/1965/labels{/name}","comments_url":"https://api.github.com/repos/prometheus/prometheus/issues/1965/comments","events_url":"https://api.github.com/repos/prometheus/prometheus/issues/1965/events","html_url":"https://github.com/prometheus/prometheus/pull/1965","id":175800805,"number":1965,"title":"federation: Collapse time series of the same name","user":{"login":"beorn7","id":5609886,"avatar_url":"https://avatars.githubusercontent.com/u/5609886?v=3","gravatar_id":"","url":"https://api.github.com/users/beorn7","html_url":"https://github.com/beorn7","followers_url":"https://api.github.com/users/beorn7/followers","following_url":"https://api.github.com/users/beorn7/following{/other_user}","gists_url":"https://api.github.com/users/beorn7/gists{/gist_id}","starred_url":"https://api.github.com/users/beorn7/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/beorn7/subscriptions","organizations_url":"https://api.github.com/users/beorn7/orgs","repos_url":"https://api.github.com/users/beorn7/repos","events_url":"https://api.github.com/users/beorn7/events{/privacy}","received_events_url":"https://api.github.com/users/beorn7/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2016-09-08T16:32:32Z","updated_at":"2016-09-27T15:37:33Z","closed_at":"2016-09-15T21:03:13Z","pull_request":{"url":"https://api.github.com/repos/prometheus/prometheus/pulls/1965","html_url":"https://github.com/prometheus/prometheus/pull/1965","diff_url":"https://github.com/prometheus/prometheus/pull/1965.diff","patch_url":"https://github.com/prometheus/prometheus/pull/1965.patch"},"body":"This will avoid duplicate MetricFamilies, thereby shrinking the size\r\nof the federation payload and also creating legal text format.\r\n\r\n@fabxc This sketches out my idea. I still have to try out how much it matters in practice (cost vs. benetif), and I will add tests if this is the way we want to go. (Actually, we should add tests anyway. As it turns out, federation is another untested territory.)"},"comment":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/comments/249903373","html_url":"https://github.com/prometheus/prometheus/pull/1965#issuecomment-249903373","issue_url":"https://api.github.com/repos/prometheus/prometheus/issues/1965","id":249903373,"user":{"login":"beorn7","id":5609886,"avatar_url":"https://avatars.githubusercontent.com/u/5609886?v=3","gravatar_id":"","url":"https://api.github.com/users/beorn7","html_url":"https://github.com/beorn7","followers_url":"https://api.github.com/users/beorn7/followers","following_url":"https://api.github.com/users/beorn7/following{/other_user}","gists_url":"https://api.github.com/users/beorn7/gists{/gist_id}","starred_url":"https://api.github.com/users/beorn7/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/beorn7/subscriptions","organizations_url":"https://api.github.com/users/beorn7/orgs","repos_url":"https://api.github.com/users/beorn7/repos","events_url":"https://api.github.com/users/beorn7/events{/privacy}","received_events_url":"https://api.github.com/users/beorn7/received_events","type":"User","site_admin":false},"created_at":"2016-09-27T15:37:33Z","updated_at":"2016-09-27T15:37:33Z","body":"I only measured the uncompressed data size. The exact shrinking ratio depends heavily on the data. One extreme is \"not at all\" if you have just one sample per metric name (AKA metric family). The other extreme is if you have a very long metric name that has a lot of samples with different short labels. (Compression ration could be arbitrarily high.)\r\n\r\nNaively, one might think, all the redundancy is compressed away by the gzip that is used by default for the federation scrape, and the savings would only stem from less (de)compression work and the less expensive subsequent extraction. The sorting of the metrics, which is happening now, could also have a beneficial impact on the compression. Sorted data compresses better than unsorted data. (Simply put: The essentially random order of metrics we had before is just noise, but gzip doesn't know that the order doesn't matter, so it cannot compress that noise away.)\r\n\r\nI haven't done any real investigation, just saying that your results might be just fine."}}
{"actor":{"display_login":"beorn7"}}