{"action":"created","issue":{"url":"https://api.github.com/repos/mapbox/cardboard/issues/184","repository_url":"https://api.github.com/repos/mapbox/cardboard","labels_url":"https://api.github.com/repos/mapbox/cardboard/issues/184/labels{/name}","comments_url":"https://api.github.com/repos/mapbox/cardboard/issues/184/comments","events_url":"https://api.github.com/repos/mapbox/cardboard/issues/184/events","html_url":"https://github.com/mapbox/cardboard/issues/184","id":174596617,"number":184,"title":"Better hash indexes","user":{"login":"mcwhittemore","id":1551510,"avatar_url":"https://avatars.githubusercontent.com/u/1551510?v=3","gravatar_id":"","url":"https://api.github.com/users/mcwhittemore","html_url":"https://github.com/mcwhittemore","followers_url":"https://api.github.com/users/mcwhittemore/followers","following_url":"https://api.github.com/users/mcwhittemore/following{/other_user}","gists_url":"https://api.github.com/users/mcwhittemore/gists{/gist_id}","starred_url":"https://api.github.com/users/mcwhittemore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcwhittemore/subscriptions","organizations_url":"https://api.github.com/users/mcwhittemore/orgs","repos_url":"https://api.github.com/users/mcwhittemore/repos","events_url":"https://api.github.com/users/mcwhittemore/events{/privacy}","received_events_url":"https://api.github.com/users/mcwhittemore/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":25,"created_at":"2016-09-01T18:44:56Z","updated_at":"2016-12-07T18:35:52Z","closed_at":null,"body":"With our current key structure `{dataset}` is the HASH and `id!{featureid}` is the RANGE. This has worked for us for a while, but recently we've noticed that we get hot indexes that can cause write failures. The root of this problem is that there can be 1000s of writes per dataset very quickly when a user is adding a lot of features.\n\nLet's use this ticket to explore other indexing options.\n\n/cc @mick @rclark \n"},"comment":{"url":"https://api.github.com/repos/mapbox/cardboard/issues/comments/265532153","html_url":"https://github.com/mapbox/cardboard/issues/184#issuecomment-265532153","issue_url":"https://api.github.com/repos/mapbox/cardboard/issues/184","id":265532153,"user":{"login":"mcwhittemore","id":1551510,"avatar_url":"https://avatars.githubusercontent.com/u/1551510?v=3","gravatar_id":"","url":"https://api.github.com/users/mcwhittemore","html_url":"https://github.com/mcwhittemore","followers_url":"https://api.github.com/users/mcwhittemore/followers","following_url":"https://api.github.com/users/mcwhittemore/following{/other_user}","gists_url":"https://api.github.com/users/mcwhittemore/gists{/gist_id}","starred_url":"https://api.github.com/users/mcwhittemore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mcwhittemore/subscriptions","organizations_url":"https://api.github.com/users/mcwhittemore/orgs","repos_url":"https://api.github.com/users/mcwhittemore/repos","events_url":"https://api.github.com/users/mcwhittemore/events{/privacy}","received_events_url":"https://api.github.com/users/mcwhittemore/received_events","type":"User","site_admin":false},"created_at":"2016-12-07T18:35:52Z","updated_at":"2016-12-07T18:35:52Z","body":"Here is a comparison of the `cardboard-hammer` scripts for one table (@jakepruitt's results from above) and for two tables.\r\n\r\nThe two table results using these shas\r\n\r\n- [cardboard](https://github.com/mapbox/cardboard/pull/186/commits/0a0263a8e56702d06bc51497e2b89adbc495c642)\r\n- [cardboard-list](https://github.com/mapbox/cardboard-list/pull/1/commits/c0384e30e57652517fc5c464e94e8596b4246a50)\r\n- [cardboard-hammer](https://github.com/mapbox/cardboard-hammer/pull/2/commits/1abe5d6df455bb192d5e27f619c3586074737aac)\r\n\r\n\r\n**$ ./bin/hammer write feature.geojson -c 150 -t 5s**\r\n\r\nFirst off, anything under 5s is still good. As expected.\r\n\r\n| Data Point |  One Table | Two Tables |\r\n| --- | --- | --- |\r\n| Size | 2158 | 2158 |\r\n| Writes | 2730 | 2211 |\r\n| Time | 5000 | 5000 |\r\n| Concurrency | 150 | 150 |\r\n| Req/s | 546 | 442.2 |\r\n| Min Latency | 200 | 280 |\r\n| Avg Latency | 276.90439560439626 | 336.03663500678437 |\r\n| Max Latency | 1691 | 782 |\r\n| Throttles | 0 | 0 |\r\n\r\n**$ ./bin/hammer write feature.geojson -c 35 -t 5m**\r\n\r\nFor longer hammers we still see throttles, but not as much. This result doesn't really tell us much though as the concurrency is set to have the one table config error as little as possible which would also result in low errors for the two table config. We'll need to run and record more concurrency to see what real improvements we have here.\r\n\r\n| Data Point |  One Table | Two Tables |\r\n| --- | --- | --- |\r\n| Size | 2158 | 2158 |\r\n| Writes | 30023 | 27222 |\r\n| Time | 300000 | 300000 |\r\n| Concurrency | 35 | 35 |\r\n| Req/s | 100.07666666666667 | 90.74 |\r\n| Min Latency | 38 | 95 |\r\n| Avg Latency | 343.45704959531133 | 382.7313202556785 |\r\n| Max Latency | 27211 | 28853 |\r\n| Throttles | 34 | 3 |\r\n\r\n\r\n**$ ./bin/hammer write-and-list feature.geojson -c 100 -t 5s**\r\n\r\nBefore there was no pause between listing requests. Since the one table result completes this in one request, I think it is safe to have added a 10ms pause between requests. This way we lower read cap usage as the table gets bigger.\r\n\r\nListing time has improved! Which makes no sense because the number of requests before getting the desired feature has tripled. It could be that I have much better network time than @jakepruitt did. It also could be I am returning ids and not full features which speeds up response time. Those two difference aside, the change in the number of requests is as expected.\r\n\r\nWe might want to run this test again on both table configs, but changing how we track time to be from after the put is done to before the list request is made. For the one table config, this will result in the request time being zero and I think it will give us a clearish idea of how long we need to wait before the feature is moved to the second table in the two table config.\r\n\r\n| Data Point |  One Table | Two Tables |\r\n| --- | --- | --- |\r\n| Size | 2158 | 2158 |\r\n| Successes | 129 | 265 |\r\n| Time | 5000 | 5000 |\r\n| Concurrency | 100 | 100 |\r\n| Min Tries | 1 | 1 |\r\n| Avg Tries | 1 | 3.113207547169819 |\r\n| Max Tries | 1 | 6 |\r\n| Min Time To List | 435 | 308 |\r\n| Avg Time To List | 1831.7209302325582 | 990.7924528301888 |\r\n| Max Time To List | 4345 | 2059 |\r\n| Read Throttles | 0 | 0 |\r\n\r\n**$ ./bin/hammer write-and-list feature.geojson -c 5 -t 5m**\r\n\r\nThis is more of the same. There is a drop in read throttles, which is too be expected as we're returning less data per read (just ids).\r\n\r\n| Data Point |  One Table | Two Tables |\r\n| --- | --- | --- |\r\n| Size | 2158 | 2158 |\r\n| Successes | 764 | 1130 |\r\n| Time | 300000 | 300000 |\r\n| Concurrency | 5 | 5 |\r\n| Min Tries | 1 | 1 |\r\n| Avg Tries | 1.0000000000000129 | 2.1061946902654536 |\r\n| Max Tries | 1 | 4 |\r\n| Min Time To List | 123 | 418 |\r\n| Avg Time To List | 1794.5785340314123 | 1022.0000000000003 |\r\n| Max Time To List | 31469 | 2183 |\r\n| Read Throttles | 2 | 0 |\r\n"}}
{"actor":{"display_login":"mcwhittemore"}}