<p>I'm setting up an ingress for the application resides in the AKS. But ran into a problem on binding the certificate to the ingress.</p>
<p>As you can see below, I am trying to reference <code>ingress-cert</code> from the KV and use it in the ingress through <code>SecretProviderClass</code></p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: {{ include &quot;secretProvider.name&quot; . }}
spec:
  provider: azure
  secretObjects:
    - secretName: ingress-tls-csi
      type: kubernetes.io/tls
      data:
        - objectName: ingress-cert
          key: tls.key
        - objectName: ingress-cert
          key: tls.crt
  parameters:
    usePodIdentity: &quot;false&quot;
    useVMManagedIdentity: &quot;true&quot;
    userAssignedIdentityID: {{ .Values.keyVault.identity }}
    keyvaultName: {{ .Values.keyVault.name }}
    objects: |
      array:
        - |
          objectName: ingress-cert
          objectType: secret
    tenantId: {{ .Values.keyVault.tenant }}
</code></pre>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include &quot;ingress.name&quot; . }}
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    kubernetes.io/ingress.allow-http: &quot;false&quot;
    appgw.ingress.kubernetes.io/override-frontend-port: &quot;443&quot;
spec:
  tls:
    - hosts:
        - {{ .Values.ingress.host }}
      secretName: ingress-tls-csi
    # Property `rules` is omitted
</code></pre>
<p>It's working fine when accessing other secrets from pods through env but for the ingress this is the output on describing it:</p>
<pre class="lang-bash prettyprint-override"><code>Name:             my-ingress
Namespace:        my-namespace
Address:          x.x.x.x
Default backend:  default-http-backend:80
TLS:
  ingress-tls-csi terminates my.example.com
Rules:
  Host                   Path  Backends
  ----                   ----  --------
Annotations:             appgw.ingress.kubernetes.io/override-frontend-port: 443
                         kubernetes.io/ingress.allow-http: false
                         kubernetes.io/ingress.class: azure/application-gateway
Events:
  Type     Reason          Age                   From                       Message
  ----     ------          ----                  ----                       -------
  Warning  SecretNotFound  3m3s (x2 over 3m10s)  azure/application-gateway  Unable to find the secret associated to secretId: [my-namespace/ingress-tls-csi]
</code></pre>
<hr />
<p>I've set up the KV integration by following the <a href="https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-driver" rel="nofollow noreferrer">Use the Azure Key Vault Provider for Secrets Store CSI Driver in an AKS cluster</a> documentation.</p>
<p>But upon following the <a href="https://learn.microsoft.com/en-us/azure/aks/csi-secrets-store-nginx-tls" rel="nofollow noreferrer">Set up Secrets Store CSI Driver to enable NGINX Ingress Controller with TLS</a> as a hint on how to implement the same with AGIC. I noticed that the certificate is added as a secret inside the AKS which is then referenced inside the ingress with <code>secretName: ingress-tls-csi</code>.</p>
<pre class="lang-bash prettyprint-override"><code>kubectl get secret -n $NAMESPACE

NAME                                             TYPE                                  DATA   AGE
ingress-tls-csi                                  kubernetes.io/tls                     2      1m34s
</code></pre>
<p>I assume that ingress can't reference the secret directly from <code>SecretProviderClass</code> as the example in the documentation need to use the <code>ingress-tls-csi</code> as a secret object which I assumed (again) created by <code>ingress-nginx</code> chart.</p>
<p>My question is how can I implement the same as the <code>ingress-nginx</code> example with AGIC?</p>
<hr />
<p>Additional information:</p>
<ul>
<li>I used AGIC with Azure CNI networking.</li>
<li>Ingress is currently working with manually added certificate with <code>kubectl</code> command. The reason I need to use the one from KV is the AKS will also be used by other people deploying under the same domain but different namespace and I think it's a bad idea to give direct access to certificate's private key.</li>
</ul>

<p>As I couldn't find a way to integrate the Ingress with the Azure Key Vault, I've implemented a workaround with GitHub Actions to retrieve the certificate and add it to the AKS. Because most of them are bash commands, the workaround isn't exclusive to GitHub Actions.</p>
<pre class="lang-yaml prettyprint-override"><code>name: Assign Cerfiticate from KV to AKS

on:
  workflow_dispatch:

jobs:
  build-push:
    name: Assign Cerfiticate from KV to AKS
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      # Azure Authentication
      - uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Authenticate and set context to AKS cluster
        run: az aks get-credentials --name &quot;my-aks-cluster&quot; --resource-group &quot;my-rg&quot; --admin

      - uses: azure/setup-helm@v3
        with:
          token: ${{ secrets.PAT }}

      # Retrieve certificate and private key from Azure Key Vault
      - name: Download certificate and private key in PKCS#12 format
        run: |
          az keyvault secret download --name my-certificate \
          --vault-name my-kv \
          --encoding base64 \
          -f certificate.pfx

      - name: Extract private key in RSA format
        run: |
          openssl pkcs12 -in certificate.pfx -nocerts -nodes -passin pass: | openssl rsa -out private.key

      - name: Extract certificate
        run: |
          openssl pkcs12 -in certificate.pfx -clcerts -nokeys -passin pass: -out certificate.crt

      - name: Deploy Kubernetes Configuration
        run: |
          helm upgrade --install release-name chart-name \
          -n my-namespace \
          --set-file mychart.ingress.key=private.key \
          --set-file mychart.ingress.certificate=certificate.crt
</code></pre>
<p>The brief explanation for the above:</p>
<ol>
<li>Download the certificate and private key in <code>.pfx</code> format</li>
<li>Use OpenSSL to extract into <code>.crt</code> and <code>.key</code></li>
<li>Pass the extracted files into <code>helm install/upgrade</code> using <code>--set-file</code></li>
</ol>
<p>Here is the secret and ingress configuration:</p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: v1
kind: Secret
metadata:
  name: ingress-tls
type: kubernetes.io/tls
data:
  tls.crt: {{ .Values.ingress.certificate | b64enc }}
  tls.key: {{ .Values.ingress.key | b64enc }}
</code></pre>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include &quot;ingress.name&quot; . }}
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    kubernetes.io/ingress.allow-http: &quot;false&quot;
    appgw.ingress.kubernetes.io/override-frontend-port: &quot;443&quot;
spec:
  tls:
    - hosts:
        - {{ .Values.ingress.host }}
      secretName: ingress-tls
  # Property `rules` is omitted
</code></pre>
<p>No additional modification to the <code>SecretProviderClass</code> is required.</p>
<p>I hope this is just a workaround because it'd be nicer if the Ingress can directly integrate with Azure Key Vault.</p>
