{"action":"opened","issue":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/4580","repository_url":"https://api.github.com/repos/prometheus/prometheus","labels_url":"https://api.github.com/repos/prometheus/prometheus/issues/4580/labels{/name}","comments_url":"https://api.github.com/repos/prometheus/prometheus/issues/4580/comments","events_url":"https://api.github.com/repos/prometheus/prometheus/issues/4580/events","html_url":"https://github.com/prometheus/prometheus/issues/4580","id":357439998,"node_id":"MDU6SXNzdWUzNTc0Mzk5OTg=","number":4580,"title":"Float rounding in recording doesn't match runtime","user":{"login":"bobrik","id":89186,"node_id":"MDQ6VXNlcjg5MTg2","avatar_url":"https://avatars0.githubusercontent.com/u/89186?v=4","gravatar_id":"","url":"https://api.github.com/users/bobrik","html_url":"https://github.com/bobrik","followers_url":"https://api.github.com/users/bobrik/followers","following_url":"https://api.github.com/users/bobrik/following{/other_user}","gists_url":"https://api.github.com/users/bobrik/gists{/gist_id}","starred_url":"https://api.github.com/users/bobrik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bobrik/subscriptions","organizations_url":"https://api.github.com/users/bobrik/orgs","repos_url":"https://api.github.com/users/bobrik/repos","events_url":"https://api.github.com/users/bobrik/events{/privacy}","received_events_url":"https://api.github.com/users/bobrik/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-09-05T23:29:19Z","updated_at":"2018-09-05T23:29:19Z","closed_at":null,"author_association":"NONE","body":"## Bug Report\r\n\r\n**What did you do?**\r\n\r\nI added a recording rule:\r\n\r\n```\r\n# Total CPU usage by mode and instance\r\n- record: instance_mode:node_cpu:sum\r\n  expr: sum(node_cpu) without (cpu)\r\n```\r\n\r\n**What did you expect to see?**\r\n\r\nRecording rule matches expression at run time, counter never goes back.\r\n\r\n**What did you see instead? Under which circumstances?**\r\n\r\nRuntime calculation:\r\n\r\n```json\r\n{\r\n  \"status\": \"success\",\r\n  \"data\": {\r\n    \"resultType\": \"matrix\",\r\n    \"result\": [\r\n      {\r\n        \"metric\": {},\r\n        \"values\": [\r\n          [\r\n            1536185100,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185130,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185160,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185190,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185220,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185250,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185280,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185310,\r\n            \"34438.13999999998\"\r\n          ],\r\n          [\r\n            1536185340,\r\n            \"34438.14999999998\"\r\n          ],\r\n          [\r\n            1536185370,\r\n            \"34438.14999999998\"\r\n          ],\r\n          [\r\n            1536185400,\r\n            \"34438.14999999998\"\r\n          ],\r\n          [\r\n            1536185430,\r\n            \"34438.14999999998\"\r\n          ],\r\n          [\r\n            1536185460,\r\n            \"34438.14999999998\"\r\n          ]\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}\r\n```\r\n\r\nRule based calculation:\r\n\r\n```json\r\n{\r\n  \"status\": \"success\",\r\n  \"data\": {\r\n    \"resultType\": \"matrix\",\r\n    \"result\": [\r\n      {\r\n        \"metric\": {\r\n          \"__name__\": \"instance_mode:node_cpu:sum\",\r\n          \"colo_id\": \"28\",\r\n          \"colo_name\": \"sea01\",\r\n          \"instance\": \"myhost\",\r\n          \"job\": \"node_exporter\",\r\n          \"mode\": \"nice\",\r\n          \"node_type\": \"metal\",\r\n          \"region\": \"wnam\"\r\n        },\r\n        \"values\": [\r\n          [\r\n            1536185100,\r\n            \"34438.139999999985\"\r\n          ],\r\n          [\r\n            1536185130,\r\n            \"34438.139999999985\"\r\n          ],\r\n          [\r\n            1536185160,\r\n            \"34438.13999999999\"\r\n          ],\r\n          [\r\n            1536185190,\r\n            \"34438.13999999999\"\r\n          ],\r\n          [\r\n            1536185220,\r\n            \"34438.13999999997\"\r\n          ],\r\n          [\r\n            1536185250,\r\n            \"34438.13999999997\"\r\n          ],\r\n          [\r\n            1536185280,\r\n            \"34438.14\"\r\n          ],\r\n          [\r\n            1536185310,\r\n            \"34438.14\"\r\n          ],\r\n          [\r\n            1536185340,\r\n            \"34438.149999999994\"\r\n          ],\r\n          [\r\n            1536185370,\r\n            \"34438.149999999994\"\r\n          ],\r\n          [\r\n            1536185400,\r\n            \"34438.14999999999\"\r\n          ],\r\n          [\r\n            1536185430,\r\n            \"34438.14999999999\"\r\n          ],\r\n          [\r\n            1536185460,\r\n            \"34438.149999999994\"\r\n          ]\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}\r\n```\r\n\r\nAs you can see, there's some rounding error. Specifically, this causes counter to jump back:\r\n\r\n```json\r\n[\r\n          [\r\n            1536185190,\r\n            \"34438.13999999999\"\r\n          ],\r\n          [\r\n            1536185220,\r\n            \"34438.13999999997\"\r\n          ],\r\n          [\r\n            1536185250,\r\n            \"34438.13999999997\"\r\n          ]\r\n]\r\n```\r\n\r\nThis makes `rate()` upset:\r\n\r\n![image](https://user-images.githubusercontent.com/89186/45126555-c2371c80-b128-11e8-9f6b-b6b9627748fc.png)\r\n\r\nGist with raw `node_cpu` data: https://gist.github.com/bobrik/048435b6e1280926c0c2cdf87bf0bc2a\r\n\r\n**Environment**\r\n\r\n* System information:\r\n\r\n```\r\n$ uname -srm\r\nLinux 4.14.59-cloudflare-2018.7.5 x86_64\r\n```\r\n\r\n* Prometheus version:\r\n\r\n```\r\n$ prometheus --version\r\nprometheus, version 2.2.1 (branch: HEAD, revision: bc6058c81272a8d938c05e75607371284236aadc)\r\n  build user:       root@149e5b3f0829\r\n  build date:       20180314-14:15:45\r\n  go version:       go1.10\r\n```\r\n"}}
{"actor":{"display_login":"bobrik"}}