{"action":"created","issue":{"url":"https://api.github.com/repos/flynn/flynn/issues/2897","repository_url":"https://api.github.com/repos/flynn/flynn","labels_url":"https://api.github.com/repos/flynn/flynn/issues/2897/labels{/name}","comments_url":"https://api.github.com/repos/flynn/flynn/issues/2897/comments","events_url":"https://api.github.com/repos/flynn/flynn/issues/2897/events","html_url":"https://github.com/flynn/flynn/issues/2897","id":158990254,"number":2897,"title":"Flynn cluster fail - discoverd flakiness?","user":{"login":"brettneese","id":504279,"avatar_url":"https://avatars.githubusercontent.com/u/504279?v=3","gravatar_id":"","url":"https://api.github.com/users/brettneese","html_url":"https://github.com/brettneese","followers_url":"https://api.github.com/users/brettneese/followers","following_url":"https://api.github.com/users/brettneese/following{/other_user}","gists_url":"https://api.github.com/users/brettneese/gists{/gist_id}","starred_url":"https://api.github.com/users/brettneese/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brettneese/subscriptions","organizations_url":"https://api.github.com/users/brettneese/orgs","repos_url":"https://api.github.com/users/brettneese/repos","events_url":"https://api.github.com/users/brettneese/events{/privacy}","received_events_url":"https://api.github.com/users/brettneese/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-06-07T18:19:48Z","updated_at":"2016-06-07T18:45:50Z","closed_at":null,"body":"Got a warning this morning that our web apps were down. Trying to extract anything from the cluster using the CLI resulted in:\r\n\r\n```GET https://controller.flynn.[domain].com/apps/dashboard/release: httpclient: raw req: unexpected status 503```\r\n\r\nI'm guessing it has to do with that `discoverd` flakiness, but not sure.\r\n\r\nflynn-host is reporting ``v20160512.0``.\r\n\r\nAny suggestions on how to get this cluster online again - or at least to a point where we can do a cluster backup? \r\n\r\nHost logs:\r\n\r\nhttps://gist.github.com/anonymous/b09a84f607c12b28c8dfde9b248e9d19\r\nhttps://gist.github.com/anonymous/2c89b9371e3e03f07c588e455e8f3626a\r\nhttps://gist.github.comanonymous//635015b1dbe7576c6ba101b29f5472cf"},"comment":{"url":"https://api.github.com/repos/flynn/flynn/issues/comments/224376470","html_url":"https://github.com/flynn/flynn/issues/2897#issuecomment-224376470","issue_url":"https://api.github.com/repos/flynn/flynn/issues/2897","id":224376470,"user":{"login":"brettneese","id":504279,"avatar_url":"https://avatars.githubusercontent.com/u/504279?v=3","gravatar_id":"","url":"https://api.github.com/users/brettneese","html_url":"https://github.com/brettneese","followers_url":"https://api.github.com/users/brettneese/followers","following_url":"https://api.github.com/users/brettneese/following{/other_user}","gists_url":"https://api.github.com/users/brettneese/gists{/gist_id}","starred_url":"https://api.github.com/users/brettneese/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brettneese/subscriptions","organizations_url":"https://api.github.com/users/brettneese/orgs","repos_url":"https://api.github.com/users/brettneese/repos","events_url":"https://api.github.com/users/brettneese/events{/privacy}","received_events_url":"https://api.github.com/users/brettneese/received_events","type":"User","site_admin":false},"created_at":"2016-06-07T18:45:50Z","updated_at":"2016-06-07T18:45:50Z","body":"Hmm, looks like you may be right:\r\n\r\n```\r\nubuntu@ip-10-0-2-77:~$ sudo zpool list\r\nNAME            SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT\r\nflynn-default  34.2G  95.5K  34.2G         -     0%     0%  1.00x  ONLINE  -\r\nubuntu@ip-10-0-2-77:~$ lsblk\r\nNAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT\r\nxvda    202:0    0  50G  0 disk \r\n└─xvda1 202:1    0  50G  0 part /```"}}
{"actor":{"display_login":"brettneese"}}