<p>We have an environment with k8s + Rancher 2 (3 nodes) and an external nginx that only forwards connections to the k8s cluster according to this documentation: <a href="https://rancher.com/docs/rancher/v2.x/en/installation/k8s-install/" rel="noreferrer">https://rancher.com/docs/rancher/v2.x/en/installation/k8s-install/</a></p>
<p>In a specific application running in this environment, when we perform a POST (since this POST takes around 3 to 4 minutes to complete), it is interrupted with the message &quot;504 Gateway Time-Out&quot; after 60 seconds. I've tried to apply specific notes to change the timeout as below, but to no avail:</p>
<p>Ingress of application:</p>
<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: api-loteamento-spring-hml
  annotations:
    nginx.ingress.kubernetes.io/proxy-connect-timeout: &quot;3600&quot;
    nginx.ingress.kubernetes.io/proxy-read-timeout: &quot;3600&quot;
    nginx.ingress.kubernetes.io/proxy-send-timeout: &quot;3600&quot;
    nginx.ingress.kubernetes.io/server-snippet: &quot;keepalive_timeout 3600s;client_body_timeout 3600s;client_header_timeout 3600s;&quot;
  labels:
    run: api-loteamento-spring-hml
spec:
  rules:
  - host: hml-api-loteamento-sp.gruposfa.bla.bla
    http:
      paths:
      - backend:
          serviceName: api-loteamento-spring-hml
          servicePort: 80
</code></pre>
<p>I have also tried to create a Global ConfigMap with the parameters as below, also without success:</p>
<pre><code>[rancher@srv-rcnode01 ssl]$ kubectl get pods -n ingress-nginx
NAME                                    READY   STATUS    RESTARTS   AGE
default-http-backend-67cf578fc4-lcz82   1/1     Running   1          38d
nginx-ingress-controller-7jcng          1/1     Running   11         225d
nginx-ingress-controller-8zxbf          1/1     Running   8          225d
nginx-ingress-controller-l527g          1/1     Running   8          225d
</code></pre>
<pre><code>[rancher@srv-rcnode01 ssl]$ kubectl get pod nginx-ingress-controller-8zxbf -n ingress-nginx -o yaml |grep configmap
    - --configmap=$(POD_NAMESPACE)/nginx-configuration
    - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
    - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
</code></pre>
<pre><code>[rancher@srv-rcnode01 ~]$ cat global-configmap.yaml
apiVersion: v1
data:
  client-body-timeout: &quot;360&quot;
  client-header-timeout: &quot;360&quot;
  proxy-connect-timeout: &quot;360&quot;
  proxy-read-timeout: &quot;360&quot;
  proxy-send-timeout: &quot;360&quot;
kind: ConfigMap
metadata:
  name: nginx-configuration
</code></pre>
<p>And apply:</p>
<pre><code>kubectl apply -f global-configmap.yaml
</code></pre>
<p>Accessing the ingress pods and checking the nginx.conf, I see that annotations are created according to the parameters set inside the application block:</p>
<pre><code>[rancher@srv-rcnode01 ~]$ kubectl -n ingress-nginx exec --stdin --tty nginx-ingress-controller-8zxbf -- /bin/bash
</code></pre>
<p>And view nginx.conf</p>
<pre><code>keepalive_timeout 3600s;client_body_timeout 3600s;client_header_timeout 3600s;

# Custom headers to proxied server
            
            
            proxy_connect_timeout                   3600s;
            proxy_send_timeout                      3600s;
            proxy_read_timeout                      3600s;
</code></pre>
<p>What I noticed at the beginning of the nginx.conf file in the &quot;server&quot; configuration block is that it has default 60-second timeout values:</p>
<pre><code># Custom headers to proxied server
            
            proxy_connect_timeout                   5s;
            proxy_send_timeout                      60s;
            proxy_read_timeout                      60s;

</code></pre>
<p>My question in this case is whether these values may be interfering with this problem, and how can I change these values in the k8s?</p>
<p>Has anyone gone through this situation or something and can give me a north?</p>
<p>Thanks!!</p>

<p>What you want to achieve was mentioned in <code>Nginx Documentation</code> in <a href="https://kubernetes.github.io/ingress-nginx/examples/customization/custom-configuration/" rel="noreferrer">Custom Configuration</a>. It recommend to use <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/" rel="noreferrer">ConfigMap</a>.</p>
<pre><code>$ cat configmap.yaml
apiVersion: v1
data:
  proxy-connect-timeout: &quot;10&quot;
  proxy-read-timeout: &quot;120&quot;
  proxy-send-timeout: &quot;120&quot;
kind: ConfigMap
metadata:
  name: ingress-nginx-controller

curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/docs/examples/customization/custom-configuration/configmap.yaml \
    | kubectl apply -f -
</code></pre>
<blockquote>
<p>If the Configmap it is updated, NGINX will be reloaded with the new configuration.</p>
</blockquote>
<p>After that, in <code>Ingress controller</code> pod, you should see entries like:</p>
<pre><code>8 controller.go:137] Configuration changes detected, backend reload required.
8 controller.go:153] Backend successfully reloaded.
</code></pre>
<p>However, please keep in mind that higher timeout values are not recommended for <code>Nginx</code>. This information can be found in <a href="https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#proxy-connect-timeout" rel="noreferrer">Nginx Ingress - proxy-connect-timeout</a>:</p>
<blockquote>
<p>Sets the timeout for establishing a connection with a proxied server. <strong>It should be noted that this timeout cannot usually exceed 75 seconds.</strong></p>
</blockquote>
<p><strong>Additional information:</strong></p>
<p><strong>1.</strong> Sometimes, when <code>Nginx Ingress</code> cannot load new configuration, you can find log like below:</p>
<pre><code>controller.go:149&quot;] - Unexpected failure reloading the backend&quot;: Invalid PID number &quot;&quot; in &quot;/tmp/nginx/pid&quot;
</code></pre>
<p>To fix it, you just need to restart <code>Ingress pod</code>.</p>
<p><strong>2.</strong> If you dont like default settings, you can create a <a href="https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/custom-template/" rel="noreferrer">config template</a> in <code>Go language</code>.</p>
