<p>I'm managing Kubernetes + nginx.</p>
<p>I'd like to install dynamic modules on nginx that are provided by Nginx Ingress Controller.
Those dynamic modules are not offered by Nginx Ingress Controller official configmap (<a href="https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/" rel="noreferrer">https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/</a>)</p>
<p>So I believe, I need to build my own Docker container of Nginx Ingress Controller.
(Could be added at this? <a href="https://github.com/kubernetes/ingress-nginx/blob/8951b7e22ad3952c549150f61d7346f272c563e1/images/nginx/rootfs/build.sh#L618-L632" rel="noreferrer">https://github.com/kubernetes/ingress-nginx/blob/8951b7e22ad3952c549150f61d7346f272c563e1/images/nginx/rootfs/build.sh#L618-L632</a> )</p>
<p>Do you know how we can customize the controller and manage it by helm chart? I'm thinking about making a Fork branch from the controller master repo on Github.
But I don't have any idea on how we install a customized version of the controller on terraform + helm chart.</p>
<p>However, I would prefer to use a non-customizable solution (because of some annotation settings)</p>
<p>Environment:
Kubernetes
Nginx Ingress Controller is installed by helm chart + terraform
Nginx Ingress Controller -&gt; <a href="https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx" rel="noreferrer">https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx</a></p>
<p>Terraform:</p>
<pre><code>resource &quot;helm_release&quot; &quot;nginx-ingress-controller&quot; {
  name      = &quot;nginx-ingress-controller&quot;
  chart     = &quot;ingress-nginx/ingress-nginx&quot;
  namespace = &quot;kube-system&quot;
  version   = &quot;3.34.0&quot;
}
</code></pre>
<p>dynamic modules
<a href="https://docs.nginx.com/nginx/admin-guide/dynamic-modules/dynamic-modules/" rel="noreferrer">https://docs.nginx.com/nginx/admin-guide/dynamic-modules/dynamic-modules/</a>
(install process might be using <code>--add-dynamic-module</code> option, and set <code>load_module modules/something.so</code> on <code>nginx.conf</code> via <code>ingress.yaml</code>)</p>
<p>Thank you.</p>

<h1>TL;DR</h1>
<p>Extend the official image with the dynamic modules, and update the <code>helm_release</code> <code>terraform</code> resource to <code>set</code> the <code>controller.image.registry</code>, <code>controller.image.image</code>, <code>controller.image.tag</code>, <code>controller.image.digest</code>, and <code>controller.image.digestChroot</code> for your custom image along with a <code>controller.config.main-snippet</code> to load the dynamic module(s) in the main context.</p>
<hr />
<p>This is similar to my previous <a href="https://stackoverflow.com/a/57741684/1423507">answer for building modules using the official nginx image</a>. You can extend the <code>ingress-nginx/controller</code> image, build the modules in one stage, extend the official image with the dynamic modules in another stage, and use the image in your <code>helm_release</code>. An example for extending the <code>ingress-nginx/controller</code> with the <a href="https://github.com/openresty/echo-nginx-module" rel="nofollow noreferrer"><code>echo-nginx-module</code></a> e.g.:</p>
<h2>Docker</h2>
<pre><code>ARG INGRESS_NGINX_CONTROLLER_VERSION
FROM registry.k8s.io/ingress-nginx/controller:${INGRESS_NGINX_CONTROLLER_VERSION} as build

ARG INGRESS_NGINX_CONTROLLER_VERSION
ENV INGRESS_NGINX_CONTROLLER_VERSION=${INGRESS_NGINX_CONTROLLER_VERSION}

USER root
RUN apk add \
        automake \
        ca-certificates \
        curl \
        gcc \
        g++ \
        make \
        pcre-dev \
        zlib-dev

RUN NGINX_VERSION=$(nginx -V 2&gt;&amp;1 |sed -n -e 's/nginx version: //p' |cut -d'/' -f2); \
    curl -L &quot;http://nginx.org/download/nginx-${NGINX_VERSION}.tar.gz&quot; | tar -C /tmp/nginx --strip-components=1 -xz

WORKDIR /src/echo-nginx-module
RUN curl -L https://github.com/openresty/echo-nginx-module/archive/refs/tags/v0.63.tar.gz | tar --strip-components=1 -xz

WORKDIR /tmp/nginx
RUN ./configure --with-compat --add-dynamic-module=/src/echo-nginx-module &amp;&amp; \
    make modules

FROM registry.k8s.io/ingress-nginx/controller:${INGRESS_NGINX_CONTROLLER_VERSION}

COPY --from=build /tmp/nginx/objs/ngx_http_echo_module.so /etc/nginx/modules/
</code></pre>
<p>... build and push the image e.g.: <code>docker build --rm -t myrepo/ingress-nginx/controller:v1.5.1-echo --build-arg INGRESS_NGINX_CONTROLLER_VERSION=v1.5.1 . &amp;&amp; docker push myrepo/ingress-nginx/controller:v1.5.1-echo</code></p>
<h2>Terraform</h2>
<p>Update the <code>terraform</code> <code>helm_release</code> resource to install the charts using the custom image and adding a <a href="https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#main-snippet" rel="nofollow noreferrer"><code>main-snippet</code></a> to set the <a href="https://nginx.org/en/docs/ngx_core_module.html#load_module" rel="nofollow noreferrer"><code>load_module</code></a> directive in the <code>main</code> context:</p>
<pre class="lang-hcl prettyprint-override"><code>resource &quot;helm_release&quot; &quot;ingress-nginx&quot; {
  name       = &quot;ingress-nginx&quot;
  namespace  = &quot;kube-system&quot;
  repository = &quot;https://kubernetes.github.io/ingress-nginx&quot;
  chart      = &quot;ingress-nginx&quot;
  version    = &quot;3.34.0&quot;
  set {
    name = &quot;controller.image.registry&quot;
    value = &quot;myrepo&quot;
  }
  set {
    name = &quot;controller.image.image&quot;
    value = &quot;ingress-nginx/controller&quot;
  }
  set {
    name = &quot;controller.image.tag&quot;
    value = &quot;v1.5.1-echo&quot;
  }
  set {
    name = &quot;controller.image.digest&quot;
    value = &quot;sha256:1b32b3e8c983ef4a32d87dead51fbbf2a2c085f1deff6aa27a212ca6beefcb72&quot;
  }
  set {
    name = &quot;controller.image.digestChroot&quot;
    value = &quot;sha256:f2e1146adeadac8eebb251284f45f8569beef9c6ec834ae1335d26617da6af2d&quot;
  }
  set {
    name = &quot;controller.config.main-snippet&quot;
    value = &lt;&lt;EOF
load_module /etc/nginx/modules/ngx_http_echo_module.so;
EOF
  }
}
</code></pre>
<p>The <code>controller.image.digest</code> is the image <code>RepoDigest</code>: <code>docker inspect myrepo/ingress-nginx/controller:v1.5.1-echo --format '{{range .RepoDigests}}{{println .}}{{end}}' |cut -d'@' -f2</code></p>
<p>The <code>controller.image.digestChroot</code> is the <code>Parent</code> sha: <code>docker inspect myrepo/ingress-nginx/controller:v1.5.1-echo --format {{.Parent}}</code></p>
<h2>Test</h2>
<ol>
<li>Create a <code>nginx</code> pod: <code>kubectl run nginx --image=nginx</code></li>
<li>Expose the pod: <code>kubectl expose pod nginx --port 80 --target-port 80</code></li>
<li>Create an ingress with a <code>server-snippet</code>:</li>
</ol>
<pre><code>cat &lt;&lt;EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/server-snippet: |
      location /hello {
        echo &quot;hello, world!&quot;;
      }
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: echo.example.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: nginx
            port:
              number: 80
  tls:
  - hosts:
    - echo.example.com
    secretName: tls-echo
EOF
</code></pre>
<blockquote>
<p>Using <a href="https://cert-manager.io/docs/" rel="nofollow noreferrer"><code>cert-manager</code></a> for TLS certificates issuance and <a href="https://github.com/kubernetes-sigs/external-dns" rel="nofollow noreferrer"><code>external-dns</code></a> for DNS management.</p>
</blockquote>
<ol start="4">
<li>Test using <code>curl</code>:</li>
</ol>
<p><a href="https://i.stack.imgur.com/ZoDrF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZoDrF.png" alt="curl echo module test on publicly exposed app" /></a></p>
