{"action":"created","issue":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/7052","repository_url":"https://api.github.com/repos/code-dot-org/code-dot-org","labels_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/7052/labels{/name}","comments_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/7052/comments","events_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/7052/events","html_url":"https://github.com/code-dot-org/code-dot-org/pull/7052","id":137418995,"number":7052,"title":"rate-limit data API requests to prevent DoS [finishes #110172562]","user":{"login":"davidsbailey","id":8001765,"avatar_url":"https://avatars.githubusercontent.com/u/8001765?v=3","gravatar_id":"","url":"https://api.github.com/users/davidsbailey","html_url":"https://github.com/davidsbailey","followers_url":"https://api.github.com/users/davidsbailey/followers","following_url":"https://api.github.com/users/davidsbailey/following{/other_user}","gists_url":"https://api.github.com/users/davidsbailey/gists{/gist_id}","starred_url":"https://api.github.com/users/davidsbailey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davidsbailey/subscriptions","organizations_url":"https://api.github.com/users/davidsbailey/orgs","repos_url":"https://api.github.com/users/davidsbailey/repos","events_url":"https://api.github.com/users/davidsbailey/events{/privacy}","received_events_url":"https://api.github.com/users/davidsbailey/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"philbogle","id":5198586,"avatar_url":"https://avatars.githubusercontent.com/u/5198586?v=3","gravatar_id":"","url":"https://api.github.com/users/philbogle","html_url":"https://github.com/philbogle","followers_url":"https://api.github.com/users/philbogle/followers","following_url":"https://api.github.com/users/philbogle/following{/other_user}","gists_url":"https://api.github.com/users/philbogle/gists{/gist_id}","starred_url":"https://api.github.com/users/philbogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philbogle/subscriptions","organizations_url":"https://api.github.com/users/philbogle/orgs","repos_url":"https://api.github.com/users/philbogle/repos","events_url":"https://api.github.com/users/philbogle/events{/privacy}","received_events_url":"https://api.github.com/users/philbogle/received_events","type":"User","site_admin":false},"milestone":null,"comments":9,"created_at":"2016-03-01T00:03:07Z","updated_at":"2016-03-02T19:16:54Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/7052","html_url":"https://github.com/code-dot-org/code-dot-org/pull/7052","diff_url":"https://github.com/code-dot-org/code-dot-org/pull/7052.diff","patch_url":"https://github.com/code-dot-org/code-dot-org/pull/7052.patch"},"body":"finishes https://www.pivotaltracker.com/story/show/110172562\r\n\r\nThis PR uses [Rack Attack](https://github.com/kickstarter/rack-attack/blob/master/README.md) backed by Redis to implement per-app rate-limiting on the APIs used by the data blocks in App Lab. The goal is to get basic rate-limiting in place ASAP to ensure availability as App Lab is unveiled at SIGCSE this week.\r\n\r\nThe per-app rate limits for each operation are as follows:\r\n\r\noperation | rate limit (req / sec)\r\n------------ | -------------\r\nreadRecords | 2\r\ncreate + update + deleteRecords | 4\r\nsetKeyValue | 4\r\ngetKeyValue | 4\r\n\r\nSome [calculations](https://docs.google.com/spreadsheets/d/1Nb571fBRqhAkZa4mfekXO0TuOjN7zTYvAVDwE04F3PA/edit#gid=0) show that these limits would require 50 badly-behaved apps to cause a DoS of our data APIs, making certain assumptions about table sizes.\r\n\r\nTested manually:\r\n- throttling works on\r\n  - readRecords\r\n  - create/update/deleteRecord\r\n  - getKeyValue\r\n  - setKeyValue\r\n- exponential backoff thresholds work\r\n- throttling in one app does not affect another app\r\n- thresholds are enforced on `/v3/` paths on both dashboard and pegasus\r\n- dashboard and pegasus both run without errors when `CDO.throttle_data_apis` is `false`\r\n\r\nNext steps (P0):\r\n- report errors to the user when running the app\r\n- add logging\r\n- add tests using FakeRedisClient\r\n"},"comment":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/comments/191380862","html_url":"https://github.com/code-dot-org/code-dot-org/pull/7052#issuecomment-191380862","issue_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/7052","id":191380862,"user":{"login":"davidsbailey","id":8001765,"avatar_url":"https://avatars.githubusercontent.com/u/8001765?v=3","gravatar_id":"","url":"https://api.github.com/users/davidsbailey","html_url":"https://github.com/davidsbailey","followers_url":"https://api.github.com/users/davidsbailey/followers","following_url":"https://api.github.com/users/davidsbailey/following{/other_user}","gists_url":"https://api.github.com/users/davidsbailey/gists{/gist_id}","starred_url":"https://api.github.com/users/davidsbailey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davidsbailey/subscriptions","organizations_url":"https://api.github.com/users/davidsbailey/orgs","repos_url":"https://api.github.com/users/davidsbailey/repos","events_url":"https://api.github.com/users/davidsbailey/events{/privacy}","received_events_url":"https://api.github.com/users/davidsbailey/received_events","type":"User","site_admin":false},"created_at":"2016-03-02T19:16:54Z","updated_at":"2016-03-02T19:16:54Z","body":"The first improvement I would make to the error message is to say \"Your app is making too many data requests. Please change your code to send fewer requests and try again in #{period} seconds.\""}}