{"action":"created","issue":{"url":"https://api.github.com/repos/istio/istio/issues/6706","repository_url":"https://api.github.com/repos/istio/istio","labels_url":"https://api.github.com/repos/istio/istio/issues/6706/labels{/name}","comments_url":"https://api.github.com/repos/istio/istio/issues/6706/comments","events_url":"https://api.github.com/repos/istio/istio/issues/6706/events","html_url":"https://github.com/istio/istio/issues/6706","id":336720487,"node_id":"MDU6SXNzdWUzMzY3MjA0ODc=","number":6706,"title":"Deploy ingressgateway as daemonset","user":{"login":"jshearer","id":4368270,"node_id":"MDQ6VXNlcjQzNjgyNzA=","avatar_url":"https://avatars0.githubusercontent.com/u/4368270?v=4","gravatar_id":"","url":"https://api.github.com/users/jshearer","html_url":"https://github.com/jshearer","followers_url":"https://api.github.com/users/jshearer/followers","following_url":"https://api.github.com/users/jshearer/following{/other_user}","gists_url":"https://api.github.com/users/jshearer/gists{/gist_id}","starred_url":"https://api.github.com/users/jshearer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jshearer/subscriptions","organizations_url":"https://api.github.com/users/jshearer/orgs","repos_url":"https://api.github.com/users/jshearer/repos","events_url":"https://api.github.com/users/jshearer/events{/privacy}","received_events_url":"https://api.github.com/users/jshearer/received_events","type":"User","site_admin":false},"labels":[{"id":735781806,"node_id":"MDU6TGFiZWw3MzU3ODE4MDY=","url":"https://api.github.com/repos/istio/istio/labels/area/networking","name":"area/networking","color":"0e8a16","default":false},{"id":572371847,"node_id":"MDU6TGFiZWw1NzIzNzE4NDc=","url":"https://api.github.com/repos/istio/istio/labels/kind/docs","name":"kind/docs","color":"fbca04","default":false},{"id":970201899,"node_id":"MDU6TGFiZWw5NzAyMDE4OTk=","url":"https://api.github.com/repos/istio/istio/labels/kind/question","name":"kind/question","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-06-28T17:56:02Z","updated_at":"2019-01-03T23:07:44Z","closed_at":null,"author_association":"NONE","body":"I'm in the process of setting up a bunch of microservices to run in GKE, and have been using ingresses with the nginx ingress controller, but wanted some of the features of istio, specifically more customizable request routing etc. The problem is, we really don't want to use `LoadBalancers` from the cloud provider (GCE), as they are painfully expensive at small scale. We've been running the ingress controller as a DaemonSet exposing ports 80 and 443, and this has worked perfectly for us so far. The issue is, I don't see an option to deploy the istio `ingressgateway` inside a `DaemonSet`. Is this supposed to be a supported solution, or do I have a conceptual misunderstanding?\r\n\r\n**Describe the solution you'd like**\r\nI'd love to be able to deploy Istio as the edge load balancer for my cluster by exposing it on every logical/physical host in the cluster.\r\n\r\nAs some extra context, we've been doing this by running the nginx ingress controller as a `DaemonSet` with the following parameters:\r\n\r\n```\r\n        ports:\r\n        - containerPort: 80\r\n          hostPort: 80\r\n          name: http\r\n          protocol: TCP\r\n        - containerPort: 443\r\n          hostPort: 443\r\n          name: https\r\n          protocol: TCP\r\n```"},"comment":{"url":"https://api.github.com/repos/istio/istio/issues/comments/451306778","html_url":"https://github.com/istio/istio/issues/6706#issuecomment-451306778","issue_url":"https://api.github.com/repos/istio/istio/issues/6706","id":451306778,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MTMwNjc3OA==","user":{"login":"FierceElements","id":21254542,"node_id":"MDQ6VXNlcjIxMjU0NTQy","avatar_url":"https://avatars2.githubusercontent.com/u/21254542?v=4","gravatar_id":"","url":"https://api.github.com/users/FierceElements","html_url":"https://github.com/FierceElements","followers_url":"https://api.github.com/users/FierceElements/followers","following_url":"https://api.github.com/users/FierceElements/following{/other_user}","gists_url":"https://api.github.com/users/FierceElements/gists{/gist_id}","starred_url":"https://api.github.com/users/FierceElements/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/FierceElements/subscriptions","organizations_url":"https://api.github.com/users/FierceElements/orgs","repos_url":"https://api.github.com/users/FierceElements/repos","events_url":"https://api.github.com/users/FierceElements/events{/privacy}","received_events_url":"https://api.github.com/users/FierceElements/received_events","type":"User","site_admin":false},"created_at":"2019-01-03T23:07:44Z","updated_at":"2019-01-03T23:07:44Z","author_association":"NONE","body":"We're also trying to deploy the `istio-ingressgateway` pods as a DaemonSet of privileged pods whose proxies bind to port 80 and 443 directly on the host. We wanted to avoid setting the `nodePorts` of the `istio-ingressgateway` service to those ports, which would require changing the `--service-node-port-range` option of the apiserver to allow such lower ports, and we don't want to give all users the ability to use privileged ports for their service `nodePorts`.\r\n\r\nWhat we're seeing is that the Envoy proxy initially binds to port 80 but not port 443:\r\n```\r\n$ kubectl -n istio-system exec -it istio-ingressgateway-6hzmq bash\r\nroot@daldevkubework02:/# netstat -nltp | grep envoy\r\ntcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      22/envoy\r\ntcp        0      0 0.0.0.0:15090           0.0.0.0:*               LISTEN      22/envoy\r\ntcp        0      0 127.0.0.1:15000         0.0.0.0:*               LISTEN      22/envoy\r\n```\r\n\r\nSometime later Envoy gets some sort of LDS update that causes it to stop listening on port 80:\r\n```\r\n$ kubectl -n istio-system logs istio-ingressgateway-6hzmq | grep _80\r\n[2019-01-03 18:26:34.015][23][info][upstream] external/envoy/source/server/lds_api.cc:80] lds: add/update listener '0.0.0.0_80'\r\n[2019-01-03 21:53:07.468][23][info][upstream] external/envoy/source/server/lds_api.cc:72] lds: remove listener '0.0.0.0_80'\r\n```\r\n\r\nHere's the relevant snippet from the `istio-ingressgateway` DaemonSet manifest:\r\n```\r\n---\r\napiVersion: extensions/v1beta1\r\nkind: DaemonSet\r\nmetadata:\r\n  name: istio-ingressgateway\r\n  namespace: istio-system\r\n  labels:\r\n    chart: gateways-1.0.4\r\n    release: istio\r\n    heritage: Tiller\r\n    app: istio-ingressgateway\r\n    istio: ingressgateway\r\nspec:\r\n  template:\r\n    metadata:\r\n      labels:\r\n        app: istio-ingressgateway\r\n        istio: ingressgateway\r\n      annotations:\r\n        sidecar.istio.io/inject: \"false\"\r\n        scheduler.alpha.kubernetes.io/critical-pod: \"\"\r\n    spec:\r\n      hostNetwork: true # ADDED\r\n      dnsPolicy: ClusterFirstWithHostNet # ADDED\r\n      serviceAccountName: istio-ingressgateway-service-account\r\n      containers:\r\n        - name: istio-proxy\r\n          image: \"docker.io/istio/proxyv2:1.0.4\"\r\n          imagePullPolicy: IfNotPresent\r\n          securityContext:\r\n            privileged: true\r\n          ports:\r\n            - containerPort: 80\r\n            - containerPort: 443\r\n            - containerPort: 31400\r\n            - containerPort: 15011\r\n            - containerPort: 8060\r\n            - containerPort: 853\r\n            - containerPort: 15030\r\n            - containerPort: 15031\r\n            - containerPort: 15090\r\n              protocol: TCP\r\n              name: http-envoy-prom\r\n```\r\n\r\nWe also changed the `istio-ingressgateway` service to `ClusterIP` so as not to expose unnecessary `nodePorts`, but changing it back to `nodePort` didn't change the above behavior.\r\n\r\nAny tips on how to troubleshoot/debug this would be appreciated."}}
{"actor":{"display_login":"FierceElements"}}