{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/57518","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/57518/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/57518/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/57518/events","html_url":"https://github.com/kubernetes/kubernetes/issues/57518","id":283930001,"number":57518,"title":"kube-proxy's default behavior of using nodeIP 127.0.0.1 when nodeIP is nil seems problematic","user":{"login":"3dinfluence","id":3537425,"avatar_url":"https://avatars2.githubusercontent.com/u/3537425?v=4","gravatar_id":"","url":"https://api.github.com/users/3dinfluence","html_url":"https://github.com/3dinfluence","followers_url":"https://api.github.com/users/3dinfluence/followers","following_url":"https://api.github.com/users/3dinfluence/following{/other_user}","gists_url":"https://api.github.com/users/3dinfluence/gists{/gist_id}","starred_url":"https://api.github.com/users/3dinfluence/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/3dinfluence/subscriptions","organizations_url":"https://api.github.com/users/3dinfluence/orgs","repos_url":"https://api.github.com/users/3dinfluence/repos","events_url":"https://api.github.com/users/3dinfluence/events{/privacy}","received_events_url":"https://api.github.com/users/3dinfluence/received_events","type":"User","site_admin":false},"labels":[{"id":128716589,"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/kube-proxy","name":"area/kube-proxy","color":"0052cc","default":false},{"id":105146071,"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/bug","name":"kind/bug","color":"e11d21","default":false},{"id":116712108,"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/sig/network","name":"sig/network","color":"d2b48c","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-12-21T16:11:46Z","updated_at":"2018-03-09T08:25:01Z","closed_at":null,"author_association":"NONE","body":"<!-- This form is for bug reports and feature requests ONLY! \r\n\r\nIf you're looking for help check [Stack Overflow](https://stackoverflow.com/questions/tagged/kubernetes) and the [troubleshooting guide](https://kubernetes.io/docs/tasks/debug-application-cluster/troubleshooting/).\r\n-->\r\n\r\n**Is this a BUG REPORT or FEATURE REQUEST?**:\r\n\r\n> Uncomment only one, leave it on its own line: \r\n>\r\n/kind bug\r\n> /kind feature\r\n\r\n\r\n**What happened**:\r\n\r\nkube-proxy came up cleanly but mis-configured iptable rules due to a mis-match in the hostname and node name within the cluster triggering a default behavior.  In this case the hostname returned the fqdn of the host and the cluster only had the hostname.  This resulted in the following warning messages when kube-proxy started.\r\n\r\n```\r\nW1220 16:46:26.875651       1 server.go:580] Failed to retrieve node info: nodes \"node07.example.com\" not found\r\nW1220 16:46:26.875785       1 proxier.go:471] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP\r\n```\r\nWhich then presents itself as pods not being able to access clusterip ip addresses with no route errors.\r\n\r\n**What you expected to happen**:\r\n\r\nWould it be better to fail to in this scenario rather than try to fall back to some default?  I spent a good amount of time trying to track down the Error messages that were a result of this before finally working back to taking a closer at the Warning message.\r\n\r\n**How to reproduce it (as minimally and precisely as possible)**:\r\n\r\nStart the kube-proxy with `--hostname-override` set to an invalid hostname should recreate this issue.\r\n\r\n**Anything else we need to know?**:\r\n\r\nWhile I think we can chalk up this issue to mis-configuration the purpose of opening this issue is really to have a conversation about the default behavior.  I've seen several other issues filed that appear to be related to this behavior: #35641, #37414, #40653.\r\n\r\n* Would throwing an error and exiting non-zero be a better default action when the nodeIP is nil?\r\n* What use case(s) does a default of `127.0.0.1` support?\r\n* If there must be a default behavior here maybe the IP address on the interface associated with the default gateway is a better default than `127.0.0.1`?\r\n* Maybe the issue isn't in the proxier code but the kube-proxy itself should error when it fails to get node info rather than a warning?\r\n\r\n**Environment**:\r\nI'm not sure how much of this is important to this but here's what my environment was when I ran into this.\r\n- Kubernetes version: 1.8.5\r\n- Cloud provider or hardware configuration: Bare metal\r\n- OS (e.g. from /etc/os-release): Centos 7 and Ubuntu 16.04"},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/371745652","html_url":"https://github.com/kubernetes/kubernetes/issues/57518#issuecomment-371745652","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/57518","id":371745652,"user":{"login":"JoshuaAndrew","id":12997509,"avatar_url":"https://avatars0.githubusercontent.com/u/12997509?v=4","gravatar_id":"","url":"https://api.github.com/users/JoshuaAndrew","html_url":"https://github.com/JoshuaAndrew","followers_url":"https://api.github.com/users/JoshuaAndrew/followers","following_url":"https://api.github.com/users/JoshuaAndrew/following{/other_user}","gists_url":"https://api.github.com/users/JoshuaAndrew/gists{/gist_id}","starred_url":"https://api.github.com/users/JoshuaAndrew/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JoshuaAndrew/subscriptions","organizations_url":"https://api.github.com/users/JoshuaAndrew/orgs","repos_url":"https://api.github.com/users/JoshuaAndrew/repos","events_url":"https://api.github.com/users/JoshuaAndrew/events{/privacy}","received_events_url":"https://api.github.com/users/JoshuaAndrew/received_events","type":"User","site_admin":false},"created_at":"2018-03-09T08:25:01Z","updated_at":"2018-03-09T08:25:01Z","author_association":"NONE","body":"@thockin   @3dinfluence  @m1093782566  @rramkumar1\r\n\r\nI ran into the issue above too, when I ran kube-proxy with  ipvs mode enabled.\r\nThis problem causes IPVS unavailability, because kube-proxy can not create kube-ipvs0 interface on host\r\n\r\n```\r\nI0309 02:20:50.634530       1 feature_gate.go:184] feature gates: map[SupportIPVSProxyMode:true]\r\nI0309 02:20:50.776657       1 server.go:391] Neither kubeconfig file nor master URL was specified. Falling back to in-cluster config.\r\nI0309 02:20:50.823405       1 server_others.go:180] Using ipvs Proxier.\r\n\r\nW0309 02:20:50.831191       1 server.go:586] Failed to retrieve node info: Get https://10.96.0.1:443/api/v1/nodes/k8s-node-2: dial tcp 10.96.0.1:443: getsockopt: connection refused\r\nW0309 02:20:50.833609       1 proxier.go:257] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP\r\n\r\nW0309 02:20:50.833657       1 proxier.go:266] IPVS scheduler not specified, use rr by default\r\nI0309 02:20:50.834225       1 server_others.go:205] Tearing down inactive rules.\r\nI0309 02:20:50.929677       1 server.go:426] Version: v1.9.2\r\n```\r\n\r\n\r\n\r\n```\r\nfunc getNodeIP(client clientset.Interface, hostname string) net.IP {\r\n\tvar nodeIP net.IP\r\n\tnode, err := client.Core().Nodes().Get(hostname, metav1.GetOptions{})\r\n\tif err != nil {\r\n\t\tglog.Warningf(\"Failed to retrieve node info: %v\", err)\r\n\t\treturn nil\r\n\t}\r\n\tnodeIP, err = utilnode.InternalGetNodeHostIP(node)\r\n\tif err != nil {\r\n\t\tglog.Warningf(\"Failed to retrieve node IP: %v\", err)\r\n\t\treturn nil\r\n\t}\r\n\treturn nodeIP\r\n}\r\n```\r\n\r\n\r\nEnvironment:\r\nI'm not sure how much of this is important to this but here's what my environment was when I ran into this.\r\n\r\n- Kubernetes version: 1.9.2\r\n- Cloud provider or hardware configuration: Bare metal\r\n- OS (e.g. from /etc/redhat-release): CentOS Linux release 7.4.1708 (Core)"}}
{"actor":{"display_login":"JoshuaAndrew"}}