{"action":"opened","issue":{"url":"https://api.github.com/repos/radiasoft/sirepo/issues/362","repository_url":"https://api.github.com/repos/radiasoft/sirepo","labels_url":"https://api.github.com/repos/radiasoft/sirepo/issues/362/labels{/name}","comments_url":"https://api.github.com/repos/radiasoft/sirepo/issues/362/comments","events_url":"https://api.github.com/repos/radiasoft/sirepo/issues/362/events","html_url":"https://github.com/radiasoft/sirepo/issues/362","id":176783239,"number":362,"title":"beta system release failure","user":{"login":"robnagler","id":5495179,"avatar_url":"https://avatars.githubusercontent.com/u/5495179?v=3","gravatar_id":"","url":"https://api.github.com/users/robnagler","html_url":"https://github.com/robnagler","followers_url":"https://api.github.com/users/robnagler/followers","following_url":"https://api.github.com/users/robnagler/following{/other_user}","gists_url":"https://api.github.com/users/robnagler/gists{/gist_id}","starred_url":"https://api.github.com/users/robnagler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robnagler/subscriptions","organizations_url":"https://api.github.com/users/robnagler/orgs","repos_url":"https://api.github.com/users/robnagler/repos","events_url":"https://api.github.com/users/robnagler/events{/privacy}","received_events_url":"https://api.github.com/users/robnagler/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-09-14T00:15:02Z","updated_at":"2016-09-14T00:15:02Z","closed_at":null,"body":"We had a beta system failure due to multiple defects in operational procedure and code. Here's a review.\r\n\r\n1. Alpha was pushed to beta\r\n2. sirepo restarted on apa11 and celery-sirepo restarted on apa14\r\n3. Transient reports were not working\r\n\r\nErrors:\r\n\r\n1. Announce that a beta release is going to happen so people can warn about potential issues \r\n2. Alpha had a db upgrade bug that was not detected. Need regression on db upgrades as well as validation against existing db #357 \r\n3. Alpha and beta use a proxy and we don't test that on dev #361\r\n4. We need automation testing for alpha #360\r\n5. Logging was completely unclear. Turn on debug mode. #359\r\n6. celery-sirepo (sequential daemon) was not restarted due to change in operating mode but not clear to whole team.\r\n7. Client was not robust in the case of an exception. Need centralized dispatch with better wrapping so we can catch as many exceptions as possible and report to server and console. #358\r\n\r\n\r\n"}}
{"actor":{"display_login":"robnagler"}}