title: Kubernetes - Ingress with gRPC
body: <p>I have a very simple grpc-server. When I expose them directly via NodePort everything works just fine. I do not have any issues with this.</p>
  <pre><code>grpc-client --&gt; NodePort --&gt; grpc-server
  </code></pre>
  <p>I changed the <code>NodePort</code> service to <code>ClusterIP</code> service and tried to use <code>ingress controller</code> to route the traffic to the grpc-server. The set up is like this. ingress and grpc-server are part of k8s cluster. grpc-client is outside - which is my local machine.</p>
  <pre><code>grpc-client --&gt; ingress --&gt; clusterip --&gt; grpc-server
  </code></pre>
  <p>My ingress is like this.</p>
  <pre><code>apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: web-ingress
    namespace: test
    annotations:
      kubernetes.io/ingress.class: &quot;nginx&quot;
      nginx.ingress.kubernetes.io/backend-protocol: &quot;GRPC&quot;
      nginx.ingress.kubernetes.io/ssl-redirect: &quot;false&quot;
      nginx.ingress.kubernetes.io/configuration-snippet: |
        proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
        grpc_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;    
  spec:
    rules:
    - http:
        paths:
        - backend:
           serviceName: grpc-server
           servicePort: 6565
  </code></pre>
  <p>Whenever I send a request, I get en exception like this in the client side.</p>
  <pre><code>io.grpc.StatusRuntimeException: INTERNAL: HTTP/2 error code: FRAME_SIZE_ERROR
  Received Rst Stream
  </code></pre>
  <p>Ingress log is like this</p>
  <pre><code>127.0.0.1 - - [28/Jul/2020:03:52:38 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - c9d46af55aa8a2b450f9f72e9d550023
  127.0.0.1 - - [28/Jul/2020:03:52:38 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.000 [] [] - - - - a9e0f02851473467fc553f2a4645377d
  127.0.0.1 - - [28/Jul/2020:03:52:40 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - fcc0668f300bd5e7e12325ae7c26fa9d
  127.0.0.1 - - [28/Jul/2020:03:52:41 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - dea2b8eb8f6884ac512654af3c2e6b47
  127.0.0.1 - - [28/Jul/2020:03:52:43 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - 0357acafce928ee05aaef1334eed1b69
  127.0.0.1 - - [28/Jul/2020:03:52:44 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - df9b75d79d9177e39a901dfa020912f2
  127.0.0.1 - - [28/Jul/2020:03:52:44 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.000 [] [] - - - - bb27acd5b86fd057af1ca8fe93f7d857
  127.0.0.1 - - [28/Jul/2020:03:52:44 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - 2ced9207652fc564bb94e0d4a14ae6fc
  127.0.0.1 - - [28/Jul/2020:03:52:45 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - b99fab13df2f64a6a0a901237dadfa27
  127.0.0.1 - - [28/Jul/2020:03:52:45 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.002 [] [] - - - - 40cb75f4e4b3df28be66044a0825f1c6
  127.0.0.1 - - [28/Jul/2020:03:52:46 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - 89cac6709278b4e54b0349075500bf74
  127.0.0.1 - - [28/Jul/2020:03:52:46 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.000 [] [] - - - - 0b032199a053069df185c620de497c90
  127.0.0.1 - - [28/Jul/2020:03:52:46 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.001 [] [] - - - - 5a07b1e24ea3e8e2066a7ed8950f7040
  127.0.0.1 - - [28/Jul/2020:03:53:46 +0000] &quot;PRI * HTTP/2.0&quot; 400 157 &quot;-&quot; &quot;-&quot; 0 0.000 [] [] - - - - 0f63eec0d0ccb9cc87c2fbb1fdfae48f
  </code></pre>
  <p>Looks like the grpc-server never received any request as I do not see any log there.</p>
  <p>What is going on?</p>