{"action":"created","issue":{"url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/issues/299","repository_url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education","labels_url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/issues/299/labels{/name}","comments_url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/issues/299/comments","events_url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/issues/299/events","html_url":"https://github.com/zooniverse/wildcam-gorongosa-education/issues/299","id":177825083,"number":299,"title":"Can't create Assignments with over 4000 Subjects","user":{"login":"shaunanoordin","id":13952701,"avatar_url":"https://avatars.githubusercontent.com/u/13952701?v=3","gravatar_id":"","url":"https://api.github.com/users/shaunanoordin","html_url":"https://github.com/shaunanoordin","followers_url":"https://api.github.com/users/shaunanoordin/followers","following_url":"https://api.github.com/users/shaunanoordin/following{/other_user}","gists_url":"https://api.github.com/users/shaunanoordin/gists{/gist_id}","starred_url":"https://api.github.com/users/shaunanoordin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shaunanoordin/subscriptions","organizations_url":"https://api.github.com/users/shaunanoordin/orgs","repos_url":"https://api.github.com/users/shaunanoordin/repos","events_url":"https://api.github.com/users/shaunanoordin/events{/privacy}","received_events_url":"https://api.github.com/users/shaunanoordin/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-09-19T16:00:25Z","updated_at":"2016-09-20T11:11:08Z","closed_at":null,"body":"Issue: When creating an Assignment that consists of over 4000 subjects, the process fails.\r\n\r\nAnalysis:\r\n* Creating an Assignment consisting of 4000 Subjects will cause the app to \"pause\" for 60 seconds before dying, with a response from the Education API saying 504 Gateway Timeout.\r\n* Creating an Assignment consisting of 3000 Subjects will \"pause\" the app for 50 seconds before a \"success\" mess is returned.\r\n* Creating an Assignment consisting of 100 Subjects will succeed after a 5 second pause. \r\n* In short:\r\n  * (1) if you have a **large number of Subjects**, it takes an **inordinately large amount of time** to submit the Create Assignment request to the Education API\r\n  * (2) if your request exceeds 60 seconds, it dies.\r\n  * (3) the cutoff appears to be at just above 3500 Subjects, but of course this depends on the connection and the Education API server availability.\r\n\r\nExperimental results:\r\n```\r\nSubjects | OK? | Size* | Response \r\n---------+-----+-------+---------\r\n100      | Yes |       |  4.90s\r\n1000     | Yes |       | 20.95s\r\n2000     | Yes |       | 38.08s\r\n3000     | Yes | 129KB | 51.40s\r\n4000     | NO  | 172KB | 60.00s - 504 Gateway Timeout\r\n10000    | NO  |       | 60.00s - 504 Gateway Timeout\r\n\r\n*Size of the Request body, if extracted as a .JSON file.\r\n```\r\n\r\nQuestion: Why does the submission process take so long? In theory an equivalent 129KB \"file upload\" shouldn't even take 50s to process.\r\n\r\nPinging @marten, @simoneduca and @eatyourgreens for your thoughts."},"comment":{"url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/issues/comments/248271696","html_url":"https://github.com/zooniverse/wildcam-gorongosa-education/issues/299#issuecomment-248271696","issue_url":"https://api.github.com/repos/zooniverse/wildcam-gorongosa-education/issues/299","id":248271696,"user":{"login":"shaunanoordin","id":13952701,"avatar_url":"https://avatars.githubusercontent.com/u/13952701?v=3","gravatar_id":"","url":"https://api.github.com/users/shaunanoordin","html_url":"https://github.com/shaunanoordin","followers_url":"https://api.github.com/users/shaunanoordin/followers","following_url":"https://api.github.com/users/shaunanoordin/following{/other_user}","gists_url":"https://api.github.com/users/shaunanoordin/gists{/gist_id}","starred_url":"https://api.github.com/users/shaunanoordin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shaunanoordin/subscriptions","organizations_url":"https://api.github.com/users/shaunanoordin/orgs","repos_url":"https://api.github.com/users/shaunanoordin/repos","events_url":"https://api.github.com/users/shaunanoordin/events{/privacy}","received_events_url":"https://api.github.com/users/shaunanoordin/received_events","type":"User","site_admin":false},"created_at":"2016-09-20T11:11:08Z","updated_at":"2016-09-20T11:11:08Z","body":"Thanks to Marten, we have an update on this bug.\r\n\r\nProblem: the Panoptes-side processing is a bottleneck.\r\n\r\nSolution: The Education API needs to return a response quickly to the user (with new \"status\" variable saying \"still processing...\"), while simultaneously processing the rest of the subjects in the background.\r\n\r\nCaveat: this means it'll take a short while before Teachers/Students can see all subjects, but this is acceptable for the Assignment use cases."}}
{"actor":{"display_login":"shaunanoordin"}}