<p>I'm trying to install the cert-manager ClusterIssuer on a AKS, and because the cluster is behind Azure Application Gateway I've gone down the route of using a DNS solver rather the HTTP. However, the challenge fails with an error calling the Cloudflare API. I've redacted emails and domains through the code snippets, the output of <code>kubectl describe challenge rabt-cert-tls-g4mcl-1991965707-2468967546</code> is:</p>
<pre><code>Events:
  Type     Reason        Age               From          Message
  ----     ------        ----              ----          -------
  Normal   Started       72s               cert-manager  Challenge scheduled for processing
  Warning  PresentError  3s (x5 over 71s)  cert-manager  Error presenting challenge: Cloudflare API Error for GET &quot;/zones?name=&lt;domain&gt;&quot;
            Error: 6003: Invalid request headers&lt;- 6103: Invalid format for X-Auth-Key header
</code></pre>
<p>I have followed the guide at <a href="https://blog.darkedges.com/2020/05/04/cert-manager-kubernetes-cloudflare-dns-update/" rel="nofollow noreferrer">https://blog.darkedges.com/2020/05/04/cert-manager-kubernetes-cloudflare-dns-update/</a> and the issues at <a href="https://github.com/jetstack/cert-manager/issues/3021" rel="nofollow noreferrer">https://github.com/jetstack/cert-manager/issues/3021</a> and <a href="https://github.com/jetstack/cert-manager/issues/2384" rel="nofollow noreferrer">https://github.com/jetstack/cert-manager/issues/2384</a> but can't see any differences beyond the apiVersion of the issuer. I've checked this against the official documentation and there are no changes from what appears in these guides.</p>
<p>The relationship between ingress and cluster issuer seems fine; if I delete and recreate the ingress a new certificate, order and challenge are created. I've verified the secret is populated and I can print it to console, so it shouldn't be sending a blank string in the header. The token is valid, I can use the example CURL request from CloudFlare to check its validity.</p>
<p>Is there somewhere I can see logs and find out exactly what is being sent?</p>
<p>ClusterIssuer</p>
<pre><code>apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-token: ${CLOUDFLARE_API_TOKEN}
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: rabt-letsencrypt
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email: &lt;email&gt;
    # ACME server URL for Let’s Encrypt’s staging environment.
    # The staging environment will not issue trusted certificates but is
    # used to ensure that the verification process is working properly
    # before moving to production
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource used to store the account's private key.
      name: rabt-letsencrypt-key
    # Enable the HTTP-01 challenge provider
    # you prove ownership of a domain by ensuring that a particular
    # file is present at the domain
    solvers:
    - dns01:
         cloudflare:
           email: &lt;email&gt;
           apiTokenSecretRef:
             name: cloudflare-api-token-secret
             key: api-key
</code></pre>
<p>Ingress</p>
<pre><code>apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rabt-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-protocol: https
    appgw.ingress.kubernetes.io/ssl-redirect: &quot;true&quot;
    cert-manager.io/cluster-issuer: rabt-letsencrypt
    cert-manager.io/acme-challenge-type: dns01
    appgw.ingress.kubernetes.io/backend-path-prefix: &quot;/&quot;
spec:
  tls:
  - hosts:
    - &quot;*.rabt.&lt;domain&gt;&quot;
    secretName: rabt-cert-tls
  rules:
  - host: &quot;mq.rabt.&lt;domain&gt;&quot;
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: rabt-mq
            port:
              number: 15672
  - host: es.rabt.&lt;domain&gt;
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: rabt-db-es-http
            port:
              number: 9200
  - host: &quot;kibana.rabt.&lt;domain&gt;&quot;
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: rabt-kb-http
            port:
              number: 5601
</code></pre>

<p>As <a href="https://stackoverflow.com/users/5525824/harsh-manvar">Harsh Manvar</a> guessed, it was an issue with the secret. I wasn't running the <code>kubectl apply</code> command through <a href="https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html" rel="nofollow noreferrer">envsubst</a> so it was encoding the literal string &quot;${CLOUDFLARE_API_TOKEN}&quot;</p>
