{"action":"created","issue":{"url":"https://api.github.com/repos/xetys/hetzner-kube/issues/139","repository_url":"https://api.github.com/repos/xetys/hetzner-kube","labels_url":"https://api.github.com/repos/xetys/hetzner-kube/issues/139/labels{/name}","comments_url":"https://api.github.com/repos/xetys/hetzner-kube/issues/139/comments","events_url":"https://api.github.com/repos/xetys/hetzner-kube/issues/139/events","html_url":"https://github.com/xetys/hetzner-kube/issues/139","id":328507834,"node_id":"MDU6SXNzdWUzMjg1MDc4MzQ=","number":139,"title":"Routing of Flannel traffic over wireguard","user":{"login":"jowenn","id":5236596,"node_id":"MDQ6VXNlcjUyMzY1OTY=","avatar_url":"https://avatars1.githubusercontent.com/u/5236596?v=4","gravatar_id":"","url":"https://api.github.com/users/jowenn","html_url":"https://github.com/jowenn","followers_url":"https://api.github.com/users/jowenn/followers","following_url":"https://api.github.com/users/jowenn/following{/other_user}","gists_url":"https://api.github.com/users/jowenn/gists{/gist_id}","starred_url":"https://api.github.com/users/jowenn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jowenn/subscriptions","organizations_url":"https://api.github.com/users/jowenn/orgs","repos_url":"https://api.github.com/users/jowenn/repos","events_url":"https://api.github.com/users/jowenn/events{/privacy}","received_events_url":"https://api.github.com/users/jowenn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-06-01T12:50:35Z","updated_at":"2018-06-06T11:32:58Z","closed_at":null,"author_association":"NONE","body":"It is nice that there is wireguard, but it is not used for inter pod communication\r\n\r\nFlannel by default uses the  device with the default route (if run on the host), I'm not sure how it is handled when run as a pod\r\n\r\nAt least I could verify that my communication between two pods on two nodes is not encrypted, because it does not use the wg0 interface.\r\n\r\nWhat I did:\r\nI created a cluster\r\nhetzner-kube cluster create ..... --ha-enabled -w 3\r\n\r\nI launched 3 nginx containers  (each container is scheduled on one node)\r\nkubectl run nginx1 --image=nginx --replicas=3 --port=80\r\n\r\nI started on each node (directly)\r\ntcpdump -nnvvSSs 1514 -i eth0 | grep GET\r\n\r\nI ran a single interactive container with curl:\r\nkubectl run curl1 --image=radial/busyboxplus:curl -i -t --rm\r\nIn this container I startet\r\ncurl http://10.244...../MYTEST1 >/dev/null\r\n\r\nFor every connection between the busy box and a container on another node I gut a nice\r\nGET /MYTEST1 HTTP/1.1 output from tcpdump\r\n \r\nTherefore the traffic is no going flannel->wg0->encryption->eth0->and on but is going directly flannel->eth0, which means the traffic is not encrypted.\r\n\r\nI also got some other GET /  for instance when kubectl connected a terminal to a pod, so it appears that also not all control plane traffic is encrypted, except etcd, which seems to really use wg0"},"comment":{"url":"https://api.github.com/repos/xetys/hetzner-kube/issues/comments/395037166","html_url":"https://github.com/xetys/hetzner-kube/issues/139#issuecomment-395037166","issue_url":"https://api.github.com/repos/xetys/hetzner-kube/issues/139","id":395037166,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NTAzNzE2Ng==","user":{"login":"pierreozoux","id":1863453,"node_id":"MDQ6VXNlcjE4NjM0NTM=","avatar_url":"https://avatars2.githubusercontent.com/u/1863453?v=4","gravatar_id":"","url":"https://api.github.com/users/pierreozoux","html_url":"https://github.com/pierreozoux","followers_url":"https://api.github.com/users/pierreozoux/followers","following_url":"https://api.github.com/users/pierreozoux/following{/other_user}","gists_url":"https://api.github.com/users/pierreozoux/gists{/gist_id}","starred_url":"https://api.github.com/users/pierreozoux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pierreozoux/subscriptions","organizations_url":"https://api.github.com/users/pierreozoux/orgs","repos_url":"https://api.github.com/users/pierreozoux/repos","events_url":"https://api.github.com/users/pierreozoux/events{/privacy}","received_events_url":"https://api.github.com/users/pierreozoux/received_events","type":"User","site_admin":false},"created_at":"2018-06-06T11:32:58Z","updated_at":"2018-06-06T11:32:58Z","author_association":"CONTRIBUTOR","body":"Yes, noticed that too.\r\n\r\nI think what is happening is that wireguard is created, then kubeadm is configured to use wireguard IPs, and then when flannel starts, it discovers interface on its own, and choose eth.\r\n\r\nI had to deploy my cluster with kubeadm (additional worker failing), and I used that config for flannel on top of wireguard:\r\nhttps://git.indie.host/indiehost/standard/blob/master/kube-flannel.yml#L74-85\r\n\r\nHope it helps!"}}
{"actor":{"display_login":"pierreozoux"}}