{"action":"created","issue":{"url":"https://api.github.com/repos/SeattleTestbed/clearinghouse/issues/174","repository_url":"https://api.github.com/repos/SeattleTestbed/clearinghouse","labels_url":"https://api.github.com/repos/SeattleTestbed/clearinghouse/issues/174/labels{/name}","comments_url":"https://api.github.com/repos/SeattleTestbed/clearinghouse/issues/174/comments","events_url":"https://api.github.com/repos/SeattleTestbed/clearinghouse/issues/174/events","html_url":"https://github.com/SeattleTestbed/clearinghouse/issues/174","id":160443840,"number":174,"title":"Explosive Logs","user":{"login":"awwad","id":13263928,"avatar_url":"https://avatars.githubusercontent.com/u/13263928?v=3","gravatar_id":"","url":"https://api.github.com/users/awwad","html_url":"https://github.com/awwad","followers_url":"https://api.github.com/users/awwad/followers","following_url":"https://api.github.com/users/awwad/following{/other_user}","gists_url":"https://api.github.com/users/awwad/gists{/gist_id}","starred_url":"https://api.github.com/users/awwad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/awwad/subscriptions","organizations_url":"https://api.github.com/users/awwad/orgs","repos_url":"https://api.github.com/users/awwad/repos","events_url":"https://api.github.com/users/awwad/events{/privacy}","received_events_url":"https://api.github.com/users/awwad/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-06-15T14:57:56Z","updated_at":"2016-06-15T16:24:29Z","closed_at":null,"body":"If the SQL server is not running, the clearinghouse's check_active_db_nodes log explodes. I'm currently looking at a 335 GB check_active_db_nodes.log file....\r\n\r\n\r\n```\r\n[2016-05-27 12:01:27.467222] CRITICAL 526751924 Unexpected exception in check_active_db_nodes.py: Traceback (most recent call last):\r\n  File \"/home/ch/deployment/clearinghouse/polling/check_active_db_nodes.py\", line 104, in main\r\n  File \"/home/ch/deployment/clearinghouse/common/api/maindb.py\", line 2198, in get_active_nodes_include_broken\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py\", line 162, in __iter__\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py\", line 965, in _fetch_all\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py\", line 238, in iterator\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py\", line 840, in execute_sql\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py\", line 79, in execute\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py\", line 64, in execute\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/utils.py\", line 97, in __exit__\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py\", line 64, in execute\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py\", line 124, in execute\r\n  File \"/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py\", line 174, in execute\r\n  File \"/usr/lib/python2.7/dist-packages/MySQLdb/connections.py\", line 36, in defaulterrorhandler\r\nOperationalError: (2006, 'MySQL server has gone away')\r\n\r\n[2016-05-27 12:01:27.467593] CRITICAL 526751924 Unexpected exception in check_active_db_nodes.py: Traceback (most recent call last):\r\n  File \"/home/ch/deployment/clearinghouse/polling/check_active_db_nodes.py\", line 104, in main\r\n  File \"/home/ch/deployment/clearinghouse/common/api/maindb.py\", line 2198, in get_active_nodes_include_broken\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py\", line 162, in __iter__\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py\", line 965, in _fetch_all\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/query.py\", line 238, in iterator\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py\", line 840, in execute_sql\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py\", line 79, in execute\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py\", line 64, in execute\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/utils.py\", line 97, in __exit__\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py\", line 64, in execute\r\n  File \"/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py\", line 124, in execute\r\n  File \"/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py\", line 174, in execute\r\n  File \"/usr/lib/python2.7/dist-packages/MySQLdb/connections.py\", line 36, in defaulterrorhandler\r\nOperationalError: (2006, 'MySQL server has gone away')\r\n```"},"comment":{"url":"https://api.github.com/repos/SeattleTestbed/clearinghouse/issues/comments/226241532","html_url":"https://github.com/SeattleTestbed/clearinghouse/issues/174#issuecomment-226241532","issue_url":"https://api.github.com/repos/SeattleTestbed/clearinghouse/issues/174","id":226241532,"user":{"login":"aaaaalbert","id":3806695,"avatar_url":"https://avatars.githubusercontent.com/u/3806695?v=3","gravatar_id":"","url":"https://api.github.com/users/aaaaalbert","html_url":"https://github.com/aaaaalbert","followers_url":"https://api.github.com/users/aaaaalbert/followers","following_url":"https://api.github.com/users/aaaaalbert/following{/other_user}","gists_url":"https://api.github.com/users/aaaaalbert/gists{/gist_id}","starred_url":"https://api.github.com/users/aaaaalbert/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aaaaalbert/subscriptions","organizations_url":"https://api.github.com/users/aaaaalbert/orgs","repos_url":"https://api.github.com/users/aaaaalbert/repos","events_url":"https://api.github.com/users/aaaaalbert/events{/privacy}","received_events_url":"https://api.github.com/users/aaaaalbert/received_events","type":"User","site_admin":false},"created_at":"2016-06-15T16:24:29Z","updated_at":"2016-06-15T16:24:29Z","body":"Oh, wow. Do you have `DEBUG = True` in your `settings.py`? (In other words, is this a problem that might bite production deployments too?)"}}
{"actor":{"display_login":"aaaaalbert"}}