{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/6400","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/6400/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/6400/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/6400/events","html_url":"https://github.com/cockroachdb/cockroach/issues/6400","id":151707219,"number":6400,"title":"storage: Optimize splitting large ranges and don't rebalance them","user":{"login":"bdarnell","id":160562,"avatar_url":"https://avatars.githubusercontent.com/u/160562?v=3","gravatar_id":"","url":"https://api.github.com/users/bdarnell","html_url":"https://github.com/bdarnell","followers_url":"https://api.github.com/users/bdarnell/followers","following_url":"https://api.github.com/users/bdarnell/following{/other_user}","gists_url":"https://api.github.com/users/bdarnell/gists{/gist_id}","starred_url":"https://api.github.com/users/bdarnell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bdarnell/subscriptions","organizations_url":"https://api.github.com/users/bdarnell/orgs","repos_url":"https://api.github.com/users/bdarnell/repos","events_url":"https://api.github.com/users/bdarnell/events{/privacy}","received_events_url":"https://api.github.com/users/bdarnell/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-04-28T18:08:09Z","updated_at":"2016-04-29T14:22:55Z","closed_at":null,"body":"On the beta cluster, we have a range that for some reason failed to split for a long time (for reasons that are still unclear). This cluster is currently configured to split at 6MB; this range had managed to grow to 6GB. Attempting to split such a huge range can cause other problems (such as blocking for a long time while it recomputes range stats). We should impose some sort of limit to ensure that if we do have trouble splitting a range, it won't continue growing without bound. For example, we could block all writes to a range that is more than 2x or 3x the configured split limit (except those writes necessary to perform a split)."},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/215733418","html_url":"https://github.com/cockroachdb/cockroach/issues/6400#issuecomment-215733418","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/6400","id":215733418,"user":{"login":"petermattis","id":6201335,"avatar_url":"https://avatars.githubusercontent.com/u/6201335?v=3","gravatar_id":"","url":"https://api.github.com/users/petermattis","html_url":"https://github.com/petermattis","followers_url":"https://api.github.com/users/petermattis/followers","following_url":"https://api.github.com/users/petermattis/following{/other_user}","gists_url":"https://api.github.com/users/petermattis/gists{/gist_id}","starred_url":"https://api.github.com/users/petermattis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petermattis/subscriptions","organizations_url":"https://api.github.com/users/petermattis/orgs","repos_url":"https://api.github.com/users/petermattis/repos","events_url":"https://api.github.com/users/petermattis/events{/privacy}","received_events_url":"https://api.github.com/users/petermattis/received_events","type":"User","site_admin":false},"created_at":"2016-04-29T14:22:55Z","updated_at":"2016-04-29T14:22:55Z","body":"@tschottdorf Yes, we already do that (compute stats for one half of the split). For a 1GB range, we'd have to compute stats on ~512MB of data. If we were able to scan that data and compute stats at ~20MB/sec (which is very optimistic), it would still take 26sec to compute the stats. Seems to me that splitting large ranges requires moving the O(size-of-range) work out of the critical path. For example, `AdminSplit` chooses a split point by scanning over the data without blocking other operations. "}}