{"action":"created","issue":{"url":"https://api.github.com/repos/cockpit-project/cockpit/issues/6359","repository_url":"https://api.github.com/repos/cockpit-project/cockpit","labels_url":"https://api.github.com/repos/cockpit-project/cockpit/issues/6359/labels{/name}","comments_url":"https://api.github.com/repos/cockpit-project/cockpit/issues/6359/comments","events_url":"https://api.github.com/repos/cockpit-project/cockpit/issues/6359/events","html_url":"https://github.com/cockpit-project/cockpit/issues/6359","id":222744358,"number":6359,"title":"Changing OpenShift TLS certificates/kube config does not work","user":{"login":"martinpitt","id":200109,"avatar_url":"https://avatars3.githubusercontent.com/u/200109?v=4","gravatar_id":"","url":"https://api.github.com/users/martinpitt","html_url":"https://github.com/martinpitt","followers_url":"https://api.github.com/users/martinpitt/followers","following_url":"https://api.github.com/users/martinpitt/following{/other_user}","gists_url":"https://api.github.com/users/martinpitt/gists{/gist_id}","starred_url":"https://api.github.com/users/martinpitt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martinpitt/subscriptions","organizations_url":"https://api.github.com/users/martinpitt/orgs","repos_url":"https://api.github.com/users/martinpitt/repos","events_url":"https://api.github.com/users/martinpitt/events{/privacy}","received_events_url":"https://api.github.com/users/martinpitt/received_events","type":"User","site_admin":false},"labels":[{"id":279643827,"url":"https://api.github.com/repos/cockpit-project/cockpit/labels/knownissue","name":"knownissue","color":"e11d21","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1643,"created_at":"2017-04-19T13:47:57Z","updated_at":"2018-05-26T06:47:23Z","closed_at":null,"author_association":"MEMBER","body":"This is a long-standing bug. Creating this issue from the [Trello card](https://trello.com/c/DalY5Iva/471-changing-tls-client-certificates-still-uses-the-old-ones-can-reproduce-this-issue-with-kube-config) to have a place where I can keep notes, and we'll use that as a known issue.\r\n\r\n### Reproducer:\r\n\r\n - Take an [old kube config](https://raw.githubusercontent.com/cockpit-project/cockpit/944eded89ae9d5229b2dbd5463218e0a63beacfd/test/verify/files/openshift.kubeconfig) with a valid but not-authorized cert and place it in your `~/.kube/config`.\r\n\r\n - Start `test/vm-run openshift` log into cockpit and go to the kubernetes dashboard. You'll get an connection error.\r\n\r\n - Put the a kubeconfig file with the correct cert in place: `scp root@10.111.112.101:/root/.kube/config ~/.kube/config` (or test/verify/files/openshift.kubeconfig from master)\r\n\r\n - Click reconnect. You still get a connection error. Browser refresh gives you the same.\r\n\r\n - Log out of cockpit and back in. And it will connect correctly.\r\n\r\n### Upstream bug:\r\nThis is a bug  in glib-networking: https://bugzilla.gnome.org/show_bug.cgi?id=781578 (patch sent there).\r\n\r\n\r\n### Old debugging log, for posterity:\r\nDebug statements show that in all cases the correct CA and client certs are being passed through and set in `parse_stream_options()` (cockpitchannel.c), i. e. `connectable->tls_cert` and `connectable->tls_database` are both correct; but the former is apparently ignored.\r\n\r\nWith broken cert:\r\n\r\n    cockpit-bridge-DEBUG: 10.111.112.101:8443: tls handshake\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: sending GET request\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: output would block\r\n    cockpit-bridge-Message: https://10.111.112.101:8443/api: Unacceptable TLS certificate: untrusted-issuer\r\n    cockpit-bridge-Message: https://10.111.112.101:8443/api: couldn't read: Unacceptable TLS certificate\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: closing stream: unknown-hostkey\r\n\r\nAfter changing to working certs, the new certificate-authority-data gets used, but the new client-certificate-data is still the old one, thus the OpenShift server does not accept the user as its cert is not signed by the new CA:\r\n\r\n    cockpit-bridge-DEBUG: 10.111.112.101:8443: tls handshake\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: sending GET request\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: output would block\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: wrote 98 bytes\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: output queue empty\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: read 198 bytes\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: response: 401 Unauthorized\r\n\r\nafter logging out and back in, both are updated:\r\n\r\n    cockpit-bridge-DEBUG: 10.111.112.101:8443: tls handshake\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: sending GET request\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: output would block\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: wrote 98 bytes\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: output queue empty\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: read 271 bytes\r\n    cockpit-bridge-DEBUG: https://10.111.112.101:8443/api: response: 200 OK\r\n\r\nWhen extracting the CA and client certs as PEM from the kube config (done with adding extra `g_debug()`, the OpenShift server replies with the same. Each command corresponds to respective case above:\r\n\r\n    curl --cert ~/broken.crt --cacert ~/authbroken.crt https://10.111.112.101:8443/api  # untrusted-issuer\r\n    curl --cert ~/broken.crt --cacert ~/authworking.crt https://10.111.112.101:8443/api  # 401 Unauthorized\r\n    curl --cert ~/working.crt --cacert ~/authworking.crt https://10.111.112.101:8443/api  # 200 OK"},"comment":{"url":"https://api.github.com/repos/cockpit-project/cockpit/issues/comments/392241465","html_url":"https://github.com/cockpit-project/cockpit/issues/6359#issuecomment-392241465","issue_url":"https://api.github.com/repos/cockpit-project/cockpit/issues/6359","id":392241465,"user":{"login":"cockpituous","id":14330603,"avatar_url":"https://avatars1.githubusercontent.com/u/14330603?v=4","gravatar_id":"","url":"https://api.github.com/users/cockpituous","html_url":"https://github.com/cockpituous","followers_url":"https://api.github.com/users/cockpituous/followers","following_url":"https://api.github.com/users/cockpituous/following{/other_user}","gists_url":"https://api.github.com/users/cockpituous/gists{/gist_id}","starred_url":"https://api.github.com/users/cockpituous/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockpituous/subscriptions","organizations_url":"https://api.github.com/users/cockpituous/orgs","repos_url":"https://api.github.com/users/cockpituous/repos","events_url":"https://api.github.com/users/cockpituous/events{/privacy}","received_events_url":"https://api.github.com/users/cockpituous/received_events","type":"User","site_admin":false},"created_at":"2018-05-26T06:47:23Z","updated_at":"2018-05-26T06:47:23Z","author_association":"CONTRIBUTOR","body":"verify/rhel-7\nOoops, it happened again<hr>\n```\n# ----------------------------------------------------------------------\n# testReconnectChangeCert (check_openshift.TestOpenshift)\n#\n/etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (C.UTF-8)\n/etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (C.UTF-8)\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nThe connection to the server 10.111.112.101:8443 was refused - did you specify the right host or port?\nNo resources found.\n\nDevTools listening on ws://127.0.0.1:9269/devtools/browser/44b0c0ea-5558-406a-8d0f-aae515c74c9f\n[0526/064414.123378:ERROR:zygote_host_impl_linux.cc(268)] Failed to adjust OOM score of renderer with pid 46639: Permission denied (13)\n[0526/064415.015879:ERROR:zygote_host_impl_linux.cc(268)] Failed to adjust OOM score of renderer with pid 46676: Permission denied (13)\n/etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (C.UTF-8)\nTraceback (most recent call last):\n  File \"/build/cockpit/bots/../test/verify/check-openshift\", line 387, in testReconnectChangeCert\n    b.wait_present(\"#service-list\")\n  File \"/build/cockpit/test/common/testlib.py\", line 270, in wait_present\n    return self.wait_js_func('ph_is_present', selector)\n  File \"/build/cockpit/test/common/testlib.py\", line 264, in wait_js_func\n    self.wait_js_cond(\"%s(%s)\" % (func, ','.join(map(jsquote, args))))\n  File \"/build/cockpit/test/common/testlib.py\", line 261, in wait_js_cond\n    self.raise_cdp_exception(\"timeout\\nwait_js_cond\", cond, result[\"exceptionDetails\"], trailer)\n  File \"/build/cockpit/test/common/testlib.py\", line 166, in raise_cdp_exception\n    raise Error(\"%s(%s): %s\" % (func, arg, msg))\nError: timeout\nwait_js_cond(ph_is_present(\"#service-list\")): condition did not become true\ncdp: {u'level': u'error', u'url': u'http://127.0.0.2:9391/cockpit/login', u'text': u'Failed to load resource: the server responded with a status of 401 (Authentication failed)', u'source': u'network', u'networkRequestId': u'46676.8', u'timestamp': 1527317055411.74}\n\nnot ok 70 testReconnectChangeCert (check_openshift.TestOpenshift) # duration: 226s\nWrote screenshot to TestOpenshift-testReconnectChangeCert-rhel-7-127.0.0.2-2501-FAIL.png\nWrote HTML dump to TestOpenshift-testReconnectChangeCert-rhel-7-127.0.0.2-2501-FAIL.html\nWrote JS log to TestOpenshift-testReconnectChangeCert-rhel-7-127.0.0.2-2501-FAIL.js.log\nJournal extracted to TestOpenshift-testReconnectChangeCert-rhel-7-127.0.0.2-2501-FAIL.log\nJournal extracted to TestOpenshift-testReconnectChangeCert-openshift-127.0.0.2-2502-FAIL.log\n/etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (C.UTF-8)\n```\nFirst occurrence: 2018-05-26T06:46:23.680537 | revision c305089a8293df25b728cc75d2275ca940a40c97, [logs](http://fedorapeople.org/groups/cockpit/logs/pull-9241-20180526-060808-c305089a-verify-rhel-7/log.html)\nTimes recorded: 1\nLatest occurrences:\n\n- 2018-05-26T06:46:23.680537 | revision c305089a8293df25b728cc75d2275ca940a40c97, [logs](http://fedorapeople.org/groups/cockpit/logs/pull-9241-20180526-060808-c305089a-verify-rhel-7/log.html)\n"}}
{"actor":{"display_login":"cockpituous"}}