{"action":"created","issue":{"url":"https://api.github.com/repos/rancher/rancher/issues/6893","repository_url":"https://api.github.com/repos/rancher/rancher","labels_url":"https://api.github.com/repos/rancher/rancher/issues/6893/labels{/name}","comments_url":"https://api.github.com/repos/rancher/rancher/issues/6893/comments","events_url":"https://api.github.com/repos/rancher/rancher/issues/6893/events","html_url":"https://github.com/rancher/rancher/issues/6893","id":193227683,"number":6893,"title":"Github auth times out, can't log in","user":{"login":"emcniece","id":1182187,"avatar_url":"https://avatars.githubusercontent.com/u/1182187?v=3","gravatar_id":"","url":"https://api.github.com/users/emcniece","html_url":"https://github.com/emcniece","followers_url":"https://api.github.com/users/emcniece/followers","following_url":"https://api.github.com/users/emcniece/following{/other_user}","gists_url":"https://api.github.com/users/emcniece/gists{/gist_id}","starred_url":"https://api.github.com/users/emcniece/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emcniece/subscriptions","organizations_url":"https://api.github.com/users/emcniece/orgs","repos_url":"https://api.github.com/users/emcniece/repos","events_url":"https://api.github.com/users/emcniece/events{/privacy}","received_events_url":"https://api.github.com/users/emcniece/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-12-02T21:57:32Z","updated_at":"2016-12-02T22:37:24Z","closed_at":null,"body":"**Rancher Version:**\r\n1.2.0\r\n\r\n**Docker Version:**\r\n1.12.3, build 6b644ec\r\n\r\n**OS and where are the hosts located? (cloud, bare metal, etc):**\r\nCoreOS, Packet.net\r\n\r\n**Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)**\r\nSingle node Rancher, internal DB\r\n\r\n**Environment Type: (Cattle/Kubernetes/Swarm/Mesos)**\r\nCattle\r\n\r\n**Steps to Reproduce:**\r\nSet up Github authorization, log out, log in\r\n\r\n**Results:** \r\nLogin action hangs for a minute, then fails. An error message is displayed in the UI: \"Error Response from Auth service\".\r\n\r\n**Expected:**\r\nLogin succeeds\r\n\r\n**Details:**\r\n\r\nThis started happening very recently after having the system running well for a few days. I am not aware of any changes within the Rancher admin, especially to the Github auth settings (and I am the only one working on this system).\r\n\r\nDuring login, http://rancher.company.ca:8080/v2-beta/token hangs for a while before returning with a JSON response:\r\n\r\n```\r\n{\"id\":\"9e67f41d-5688-4677-884d-fc79a0340266\",\"type\":\"error\",\"links\":{},\"status\":503,\"code\":\"AuthError\",\"message\":\"Error Response from Auth service\",\"detail\":\"Status code from Auth Service: 500\",\"baseType\":\"error\",\"actionLinks\":{}}\r\n```\r\n\r\nRancher logs reveal IO timeout:\r\n\r\n```\r\ntime=\"2016-12-02T21:53:09Z\" level=error msg=\"Failed to pull the catalog from git repo https://git.rancher.io/infra-catalog.git, error: exit status 1\"\r\ntime=\"2016-12-02T21:53:09Z\" level=error msg=\"Received error from github: Post https://github.com/login/oauth/access_token: dial tcp 192.30.253.112:443: i/o timeout\"\r\ntime=\"2016-12-02T21:53:09Z\" level=error msg=\"Github getAccessToken: GET url https://github.com/login/oauth/access_token received error from github, err: Post https://github.com/login/oauth/access_token: dial tcp 192.30.253.112:443: i/o timeout\"\r\ntime=\"2016-12-02T21:53:09Z\" level=error msg=\"Error generating accessToken from github Post https://github.com/login/oauth/access_token: dial tcp 192.30.253.112:443: i/o timeout\"\r\ntime=\"2016-12-02T21:53:09Z\" level=error msg=\"GetToken failed with error: Post https://github.com/login/oauth/access_token: dial tcp 192.30.253.112:443: i/o timeout\"\r\n2016-12-02 21:53:09,586 ERROR [:] [] [] [] [tp1830712962-16] [.a.a.i.e.ExternalServiceAuthProvider] Got error from Auth service. statusCode: 500\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/rancher/rancher/issues/comments/264581387","html_url":"https://github.com/rancher/rancher/issues/6893#issuecomment-264581387","issue_url":"https://api.github.com/repos/rancher/rancher/issues/6893","id":264581387,"user":{"login":"emcniece","id":1182187,"avatar_url":"https://avatars.githubusercontent.com/u/1182187?v=3","gravatar_id":"","url":"https://api.github.com/users/emcniece","html_url":"https://github.com/emcniece","followers_url":"https://api.github.com/users/emcniece/followers","following_url":"https://api.github.com/users/emcniece/following{/other_user}","gists_url":"https://api.github.com/users/emcniece/gists{/gist_id}","starred_url":"https://api.github.com/users/emcniece/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/emcniece/subscriptions","organizations_url":"https://api.github.com/users/emcniece/orgs","repos_url":"https://api.github.com/users/emcniece/repos","events_url":"https://api.github.com/users/emcniece/events{/privacy}","received_events_url":"https://api.github.com/users/emcniece/received_events","type":"User","site_admin":false},"created_at":"2016-12-02T22:37:24Z","updated_at":"2016-12-02T22:37:24Z","body":"Very interesting, thanks for the pointer. Seems like this might be a networking issue.\r\n\r\n```\r\nroot@b3dd3643d47f:/# curl -v https://github.com/login/oauth/access_token\r\n* Hostname was NOT found in DNS cache\r\n*   Trying 192.30.253.113...\r\n* connect to 192.30.253.113 port 443 failed: Connection timed out\r\n*   Trying 192.30.253.112...\r\n* After 86278ms connect time, move on!\r\n* connect to 192.30.253.112 port 443 failed: Connection timed out\r\n* Failed to connect to github.com port 443: Connection timed out\r\n* Closing connection 0\r\ncurl: (7) Failed to connect to github.com port 443: Connection timed out\r\n\r\nroot@b3dd3643d47f:/# curl --version\r\ncurl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3\r\nProtocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp\r\nFeatures: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP\r\n\r\nroot@b3dd3643d47f:/# uname -a\r\nLinux b3dd3643d47f 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 x86_64 x86_64 GNU/Linux\r\n```"}}
{"actor":{"display_login":"emcniece"}}