<p>We make use of Ingress to create HTTPS load balancers that forward directly to our (typically nodejs) services. However, recently we have wanted more control of traffic in front of nodejs which the Google load balancer doesn't provide.</p>

<ul>
<li>Standardised, custom error pages</li>
<li>Standard rewrite rules (e.g redirect http to https)</li>
<li>Decouple pod readinessProbes from load balancer health checks (so we can still serve custom error pages when there are no healthy pods).</li>
</ul>

<p>We use nginx in other parts of our stack so this seems like a good choice, and I have seen several examples of nginx being used to front services in Kubernetes, typically in one of two configurations.</p>

<ul>
<li>An nginx container in every pod forwarding traffic directly to the application on localhost.</li>
<li>A separate nginx Deployment &amp; Service, scaled independently and forwarding traffic to the appropriate Kubernetes Service.</li>
</ul>

<p>What are the pros/cons of each method and how should I determine which one is most appropriate for our use case?</p>

<p>Following on from Vincent H, I'd suggest pipelining the Google HTTPS Load Balancer to an nginx ingress controller.</p>

<p>As you've mentioned, this can scale independently; have it's own health checks; and you can standardise your error pages. </p>

<p>We've achieved this by having a single <code>kubernetes.io/ingress.class: "gce"</code> ingress object, which has a default backend of our nginx ingress controller. All our other ingress objects are annotated with <code>kubernetes.io/ingress.class: "nginx"</code>. </p>

<p>We're using the controller documented here: <a href="https://github.com/kubernetes/ingress/tree/master/controllers/nginx" rel="nofollow noreferrer">https://github.com/kubernetes/ingress/tree/master/controllers/nginx</a>. With a <a href="https://github.com/kubernetes/ingress/blob/dd7f8b4a972d9c7346da2b37419ff3529d3b38d0/controllers/nginx/configuration.md" rel="nofollow noreferrer">custom <code>/etc/nginx/template/nginx.tmpl</code></a> allowing complete control over the ingress.</p>

<blockquote>
  <p>For complete transparency, we haven't (yet) setup custom error pages in the nginx controller, however the <a href="https://github.com/kubernetes/ingress/tree/dd7f8b4a972d9c7346da2b37419ff3529d3b38d0/examples/customization/custom-errors/nginx" rel="nofollow noreferrer">documentation</a> appears straight forward.</p>
</blockquote>
