{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5518","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5518/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5518/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5518/events","html_url":"https://github.com/cockroachdb/cockroach/issues/5518","id":143075260,"number":5518,"title":"storage: Separate out consistency checks over floating point data","user":{"login":"vivekmenezes","id":11774212,"avatar_url":"https://avatars.githubusercontent.com/u/11774212?v=3","gravatar_id":"","url":"https://api.github.com/users/vivekmenezes","html_url":"https://github.com/vivekmenezes","followers_url":"https://api.github.com/users/vivekmenezes/followers","following_url":"https://api.github.com/users/vivekmenezes/following{/other_user}","gists_url":"https://api.github.com/users/vivekmenezes/gists{/gist_id}","starred_url":"https://api.github.com/users/vivekmenezes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vivekmenezes/subscriptions","organizations_url":"https://api.github.com/users/vivekmenezes/orgs","repos_url":"https://api.github.com/users/vivekmenezes/repos","events_url":"https://api.github.com/users/vivekmenezes/events{/privacy}","received_events_url":"https://api.github.com/users/vivekmenezes/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"vivekmenezes","id":11774212,"avatar_url":"https://avatars.githubusercontent.com/u/11774212?v=3","gravatar_id":"","url":"https://api.github.com/users/vivekmenezes","html_url":"https://github.com/vivekmenezes","followers_url":"https://api.github.com/users/vivekmenezes/followers","following_url":"https://api.github.com/users/vivekmenezes/following{/other_user}","gists_url":"https://api.github.com/users/vivekmenezes/gists{/gist_id}","starred_url":"https://api.github.com/users/vivekmenezes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vivekmenezes/subscriptions","organizations_url":"https://api.github.com/users/vivekmenezes/orgs","repos_url":"https://api.github.com/users/vivekmenezes/repos","events_url":"https://api.github.com/users/vivekmenezes/events{/privacy}","received_events_url":"https://api.github.com/users/vivekmenezes/received_events","type":"User","site_admin":false},"milestone":null,"comments":14,"created_at":"2016-03-23T20:44:13Z","updated_at":"2016-03-24T15:38:50Z","closed_at":null,"body":"We run periodic consistency checks over all the replicated data. Timeseries data which is cockroach internal, is replicated consistently, but merged independently often outside of raft on each replica. While the consistency checker forces a merge on each replica at a snapshot, thereby making it consistent before computing a checksum on the data, the data itself contains floating points, with the merge operator running floating point math like adding up all the timeseries values. We'd like to continue running consistency checks on this data so we can catch consistency problems on homogenous clusters, while accepting consistency differences on heterogenous one.\r\n\r\nOptions:\r\n1. Separate out the checksum over timeseries data so that it doesn't interfere with the range checksum (preferred).\r\n2. Convert the floating point values into integers and run integer math on them. Not recommended because we also really don't care about this data being that accurate."},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/200891114","html_url":"https://github.com/cockroachdb/cockroach/issues/5518#issuecomment-200891114","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5518","id":200891114,"user":{"login":"tamird","id":1535036,"avatar_url":"https://avatars.githubusercontent.com/u/1535036?v=3","gravatar_id":"","url":"https://api.github.com/users/tamird","html_url":"https://github.com/tamird","followers_url":"https://api.github.com/users/tamird/followers","following_url":"https://api.github.com/users/tamird/following{/other_user}","gists_url":"https://api.github.com/users/tamird/gists{/gist_id}","starred_url":"https://api.github.com/users/tamird/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tamird/subscriptions","organizations_url":"https://api.github.com/users/tamird/orgs","repos_url":"https://api.github.com/users/tamird/repos","events_url":"https://api.github.com/users/tamird/events{/privacy}","received_events_url":"https://api.github.com/users/tamird/received_events","type":"User","site_admin":false},"created_at":"2016-03-24T15:38:50Z","updated_at":"2016-03-24T15:38:50Z","body":"> We don't haven't seen any inconsistencies due to float. But this issue is about ensuring that when cockroach runs on heterogenous clusters, and inconsistencies are seen due to rounding error, or any other issue caused by FP, the consistency checker doesn't panic. I'd like to take values that we believe might become inconsistent out, while continuing to checksum them so that if we notice a true consistency issue on a homogenous system we notice.\r\n\r\nI don't follow this - are you suggesting that FP operations are not platform-independent? What \"other issues caused by FP\" do we expect?"}}