<p>I'm deploying cert-manager on Azure AKS and trying to have it request a Let's Encrypt certificate. It fails with <code>certificate signed by unknown authority</code> error and I have problem troubleshooting it further.</p>

<p>Not sure whether this is a problem with trusting LE server, a <code>tunnelfront</code> pod, or maybe an internal AKS self-generated CA. So my questions would be:</p>

<ul>
<li>how to force cert-manager to debug (display more info) regarding the certificate it does not trust?</li>
<li>maybe the problem is occuring regularly and there is a known solution?</li>
<li>what steps should be undertaken to debug the issue further?</li>
</ul>

<p>I have created an issue on <code>jetstack/cert-manager</code>'s Github page, but was not answered, so I came here.</p>

<p>The whole story is as follows:</p>

<p>Certificates are not created. The following errors are reported:</p>

<p>the certificate:
<code>Error from server: conversion webhook for &amp;{map[apiVersion:cert-manager.io/v1alpha2 kind:Certificate metadata:map[creationTimestamp:2020-05-13T17:30:48Z generation:1 name:xxx-tls namespace:test ownerReferences:[map[apiVersion:extensions/v1beta1 blockOwnerDeletion:true controller:true kind:Ingress name:xxx-ingress uid:6d73b182-bbce-4834-aee2-414d2b3aa802]] uid:d40bc037-aef7-4139-868f-bd615a423b38] spec:map[dnsNames:[xxx.test.domain.com] issuerRef:map[group:cert-manager.io kind:ClusterIssuer name:letsencrypt-prod] secretName:xxx-tls] status:map[conditions:[map[lastTransitionTime:2020-05-13T18:55:31Z message:Waiting for CertificateRequest "xxx-tls-1403681706" to complete reason:InProgress status:False type:Ready]]]]} failed: Post https://cert-manager-webhook.cert-manager.svc:443/convert?timeout=30s: x509: certificate signed by unknown authority</code></p>

<p>cert-manager-webhook container:
<code>cert-manager 2020/05/15 14:22:58 http: TLS handshake error from 10.20.0.19:35350: remote error: tls: bad certificate</code></p>

<p>Where <code>10.20.0.19</code> is the IP of <code>tunnelfront</code> pod.</p>

<p>Debugging with <a href="https://cert-manager.io/docs/faq/acme/" rel="nofollow noreferrer">https://cert-manager.io/docs/faq/acme/</a> sort of "fails" when trying to <code>kubectl describe order...</code> as <code>kubectl describe certificaterequest...</code> returns CSR contents with error (as above), but not the order ID.</p>

<p>Environment details:</p>

<ul>
<li>Kubernetes version: <code>1.15.10</code></li>
<li>Cloud-provider/provisioner : <code>Azure (AKS)</code></li>
<li>cert-manager version: <code>0.14.3</code></li>
<li>Install method: static manifests (see below) + cluster issuer (see below) + regular CRDs (not legacy)</li>
</ul>

<p>cluster issuer:</p>

<pre><code>kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  namespace: cert-manager
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: x
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - dns01:
          azuredns:
            clientID: x
            clientSecretSecretRef:
              name: cert-manager-stage
              key: CLIENT_SECRET
            subscriptionID: x
            tenantID: x
            resourceGroupName: dns-stage
            hostedZoneName: x
</code></pre>

<p>the manifest:</p>

<pre><code>  imagePullSecrets: []
  isOpenshift: false

  priorityClassName: ""
  rbac:
    create: true

  podSecurityPolicy:
    enabled: false

  logLevel: 2

  leaderElection:
    namespace: "kube-system"

replicaCount: 1

strategy: {}


image:
  repository: quay.io/jetstack/cert-manager-controller
  pullPolicy: IfNotPresent

  tag: v0.14.3

clusterResourceNamespace: ""

serviceAccount:
  create: true
  name:
  annotations: {}

extraArgs: []

extraEnv: []

resources: {}

securityContext:
  enabled: false
  fsGroup: 1001
  runAsUser: 1001

podAnnotations: {}

podLabels: {}

nodeSelector: {}

ingressShim:
  defaultIssuerName: letsencrypt-prod
  defaultIssuerKind: ClusterIssuer

prometheus:
  enabled: true
  servicemonitor:
    enabled: false
    prometheusInstance: default
    targetPort: 9402
    path: /metrics
    interval: 60s
    scrapeTimeout: 30s
    labels: {}


affinity: {}

tolerations: []

webhook:
  enabled: true
  replicaCount: 1

  strategy: {}

  podAnnotations: {}

  extraArgs: []

  resources: {}

  nodeSelector: {}

  affinity: {}

  tolerations: []

  image:
    repository: quay.io/jetstack/cert-manager-webhook
    pullPolicy: IfNotPresent
    tag: v0.14.3

  injectAPIServerCA: true

  securePort: 10250

cainjector:
  replicaCount: 1

  strategy: {}

  podAnnotations: {}

  extraArgs: []

  resources: {}

  nodeSelector: {}

  affinity: {}

  tolerations: []

  image:
    repository: quay.io/jetstack/cert-manager-cainjector
    pullPolicy: IfNotPresent
    tag: v0.14.3
</code></pre>

theo
<p>Seems that <code>v0.14.3</code> had a bug of some sort. The problem does not occur for <code>v0.15.0</code>.</p>
