{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kops/issues/800","repository_url":"https://api.github.com/repos/kubernetes/kops","labels_url":"https://api.github.com/repos/kubernetes/kops/issues/800/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kops/issues/800/comments","events_url":"https://api.github.com/repos/kubernetes/kops/issues/800/events","html_url":"https://github.com/kubernetes/kops/issues/800","id":187131488,"number":800,"title":"Support internal DNS names when exporting kubecfg","user":{"login":"jschneiderhan","id":369108,"avatar_url":"https://avatars.githubusercontent.com/u/369108?v=3","gravatar_id":"","url":"https://api.github.com/users/jschneiderhan","html_url":"https://github.com/jschneiderhan","followers_url":"https://api.github.com/users/jschneiderhan/followers","following_url":"https://api.github.com/users/jschneiderhan/following{/other_user}","gists_url":"https://api.github.com/users/jschneiderhan/gists{/gist_id}","starred_url":"https://api.github.com/users/jschneiderhan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jschneiderhan/subscriptions","organizations_url":"https://api.github.com/users/jschneiderhan/orgs","repos_url":"https://api.github.com/users/jschneiderhan/repos","events_url":"https://api.github.com/users/jschneiderhan/events{/privacy}","received_events_url":"https://api.github.com/users/jschneiderhan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-11-03T17:06:54Z","updated_at":"2016-11-04T14:05:28Z","closed_at":null,"body":"When I created my cluster, I specified the `--admin-access` flag to the CIDR of a management subnet in my VPC. I connect to the management subnet using a VPN server, which gives me network access to the machines in my cluster.\r\n\r\nWhen I run a `kops export kubecfg`, it populates my `~/.kube.config` using the public DNS entry for my cluster instead, which isn't reachable from my workstation. If I change the `server` for the cluster in my `~/.kube/config` file to the \"internal\" DNS name that was create for the cluster, everything works. Unfortuneatly, every time I update the cluster, the `~/.kube/config` file resets the `server` attribute to the public DNS name.\r\n\r\nWould it make sense to have a flag to support using the internal hostname in `~/.kube/config`?\r\n\r\nIf so, I'd be happy to give implementing it a shot, although I may need a few hints on implementation ideas. \r\n\r\nIf not, is there a recommended way to setup kubectl when accessing the cluster over a private network? I'd be happy to submit a PR with some documentation updates if someone wants to help me explain the approach."},"comment":{"url":"https://api.github.com/repos/kubernetes/kops/issues/comments/258439935","html_url":"https://github.com/kubernetes/kops/issues/800#issuecomment-258439935","issue_url":"https://api.github.com/repos/kubernetes/kops/issues/800","id":258439935,"user":{"login":"jschneiderhan","id":369108,"avatar_url":"https://avatars.githubusercontent.com/u/369108?v=3","gravatar_id":"","url":"https://api.github.com/users/jschneiderhan","html_url":"https://github.com/jschneiderhan","followers_url":"https://api.github.com/users/jschneiderhan/followers","following_url":"https://api.github.com/users/jschneiderhan/following{/other_user}","gists_url":"https://api.github.com/users/jschneiderhan/gists{/gist_id}","starred_url":"https://api.github.com/users/jschneiderhan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jschneiderhan/subscriptions","organizations_url":"https://api.github.com/users/jschneiderhan/orgs","repos_url":"https://api.github.com/users/jschneiderhan/repos","events_url":"https://api.github.com/users/jschneiderhan/events{/privacy}","received_events_url":"https://api.github.com/users/jschneiderhan/received_events","type":"User","site_admin":false},"created_at":"2016-11-04T14:05:28Z","updated_at":"2016-11-04T14:05:28Z","body":"A bit more information about my setup:\r\n\r\nI have a few VPCs, all of which existing before I spun up my k8s cluster. One is a management network (172.16.0.0/16) which has an OpenVPN server and a few other monitoring servers in it. The other VPCs are just for different environments (dev/staging/production). Most of our servers are in private subnets without public IPs, with most internet traffic coming though ELBs in public subnets.\r\n\r\nThe management network has VPC peering connections with the other VPCs, and the OpenVPN server sets up routes to the other VPCs for clients. So when we have a VPN connection established, we can connect to the machines from our workstations via the local IP. It works pretty well, and has been so much easier to manage than our previous solution of using jump boxes.  If an engineer leaves or their account is compromised or something we can just cut network access off at the VPN server immediately before disable all their other accounts/credentials/other lose ends.\r\n\r\nNone of that has anything to do with k8s, but I figured it may be useful background info.\r\n\r\nWhen I went to create my staging k8s cluster, my first thought was that it would be great to put all of the masters and nodes into private subnets without public IPs and all of us engineers can just access the cluster using local IPs over the VPN connection. That would keep all of our k8s instances from being routable over the internet, and have the only incoming internet traffic be though ELBs in public subnets create by k8s services.\r\n\r\nThat isn't quite possible yet, but I know people are putting awesome work into it (#428)! I figured the default approach of putting the k8s instances in public subnets with public IPs and hostnames was fine for the time being, especially if we could limit access to the k8s api to just local traffic from our management network. I don't really want the k8s api directly accessible over the internet, if possible. It turned out that kops has the `--admin-access` flag for just that purpose, so I used it when creating my cluster.\r\n\r\nThis is the command that I used to create my cluster (changed a identifiers)\r\n\r\n```\r\nkops create cluster --admin-access 172.16.0.0/16 \\\r\n                    --cloud aws \\\r\n                    --master-zones us-east-1a,us-east-1b,us-east-1c \\\r\n                    --network-cidr 172.20.0.0/16 \\\r\n                    --node-count 3 \\\r\n                    --vpc vpc-xxxxxx \\\r\n                    --zones us-east-1a,us-east-1b,us-east-1c \\\r\n                    --name k8s.example.com \\\r\n                    --ssh-public-key ~/.ssh/example.pub\r\n```\r\n\r\nEverything came up flawlessly. I actually couldn't believe it. I had just spent the past two weeks working through \"kubernetes the hard way\" setting everything up manually and tweaking things, then dealing with the fallout of my tweaks, so the fact that a tool did all of that in a matter of minutes was amazing.\r\n\r\nAnyway, the only hiccup I had was that when I exported my kubectl config and tried running a command, I couldn't connect to the API. Upon closer inspection, I noticed that my `~/.kube/config` file was using `api.k8s.example.com` as the server address. `api.k8s.example.com` resolves to the public IP addresses of the master servers. The k8s api is not accessible via the public IP addresses because the --admin-access flag adds a security group rule to limit access to 172.16.0.0/16.\r\n\r\nIn Route53 I saw that there is a corresponding DNS entry setup for `api.internal.example.com` which resolves to the private ip addresses of the master servers. When I updated my `~/.kube/config` file to use the internal DNS entry, everything worked. Now all k8s api traffic is limited to VPN traffic and not accessible via the open internet.\r\n\r\nSo all of that works, but my `~/.kube/config` keeps getting reset to the public DNS entry each time I use kops to update my cluster, so I have to go in and edit by hand each time. No big deal, but it would be great if it was supported by kops.\r\n\r\nI'm not sure how kops can get the internal name when generating the config, must it must be available somewhere, since it's creating it in the first place! I'll poke around a bit and see if I can't figure it out. I do know that if I run `kops edit cluster k8s.example.com` there is an attribute for `masterPublicName: api.k8s.example.com`, but no corresponding private attribute. Perhaps an implementation idea would be to store the internal name alongside wherever that public name is being stored (S3 presumably) then provide a flag to allow specification during `kops export kubecfg` of whether to use the public (default) or private hostname?"}}
{"actor":{"display_login":"jschneiderhan"}}