<p>I am trying to create a kubernetes cluster namespace with auto generated DNS for ingress, secured with <a href="https://letsencrypt.org/" rel="nofollow noreferrer">Let's Encrypt</a> TLS certificates. Unfortunately i'm running in some trouble and do not know where to look for the solution.</p>

<p>Deployment is being done with a multi-stage yaml pipeline into an AKS cluster, i've setup an nginx ingress controller and cert-manager, both in a separate namespace. The deployment succeeds and everything seems to be running, but the exposed hostnames from the ingress are not reachable. When taking a look at the certificates i see the following</p>

<pre><code>Name:         letsencrypt-tls-cd
Namespace:    myApp-dev
Labels:       app.kubernetes.io/instance=myApp
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=cd
              app.kubernetes.io/version=9.3.0
              helm.sh/chart=cd-1.0.0
Annotations:  &lt;none&gt;
API Version:  cert-manager.io/v1alpha3
Kind:         Certificate
Metadata:
  Creation Timestamp:  2020-06-15T11:59:53Z
  Generation:          1
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  myApp-cd
    UID:                   a6cbbf69-749e-4dd1-81cc-37a817051690
  Resource Version:        1218430
  Self Link:               /apis/cert-manager.io/v1alpha3/namespaces/myApp-dev/certificates/letsencrypt-tls-cd
  UID:                     46ac0acb-71bf-4dbc-a376-c024e92d68ca
Spec:
  Dns Names:
    cd-myApp-dev.dev
  Issuer Ref:
    Group:      cert-manager.io
    Kind:       Issuer
    Name:       letsencrypt-prod
  Secret Name:  letsencrypt-tls-cd
Status:
  Conditions:
    Last Transition Time:  2020-06-15T11:59:53Z
    Message:               ***Waiting for CertificateRequest "letsencrypt-tls-cd-95531636" to complete***
    Reason:                InProgress
    Status:                False
    Type:                  Ready
Events:
  Type    Reason        Age   From          Message
  ----    ------        ----  ----          -------
  Normal  GeneratedKey  57m   cert-manager  Generated a new private key
  Normal  Requested     57m   cert-manager  Created new CertificateRequest resource "letsencrypt-tls-cd-95531636"
</code></pre>

<p>Looking into the certificate request :</p>

<pre><code>Name:         letsencrypt-tls-cd-95531636
Namespace:    myApp-dev
Labels:       app.kubernetes.io/instance=myApp
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=cd
              app.kubernetes.io/version=9.3.0
              helm.sh/chart=cd-1.0.0
Annotations:  cert-manager.io/certificate-name: letsencrypt-tls-cd
              cert-manager.io/private-key-secret-name: letsencrypt-tls-cd
API Version:  cert-manager.io/v1alpha3
Kind:         CertificateRequest
Metadata:
  Creation Timestamp:  2020-06-15T11:59:54Z
  Generation:          1
  Owner References:
    API Version:           cert-manager.io/v1alpha2
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Certificate
    Name:                  letsencrypt-tls-cd
    UID:                   46ac0acb-71bf-4dbc-a376-c024e92d68ca
  Resource Version:        1218442
  Self Link:               /apis/cert-manager.io/v1alpha3/namespaces/myApp-dev/certificaterequests/letsencrypt-tls-cd-95531636
  UID:                     2bef5e93-6722-43c0-bd2c-283d70334b1c
Spec:
  Csr:  mySecret
  Issuer Ref:
    Group:  cert-manager.io
    Kind:   Issuer
    Name:   letsencrypt-prod
Status:
  Conditions:
    Last Transition Time:  2020-06-15T11:59:54Z
    Message:               Waiting on certificate issuance from order myApp-dev/letsencrypt-tls-cd-95531636-1679437339: "pending"
    Reason:                Pending
    Status:                False
    Type:                  Ready
Events:
  Type    Reason        Age   From          Message
  ----    ------        ----  ----          -------
  Normal  OrderCreated  58m   cert-manager  Created Order resource myApp-dev/letsencrypt-tls-cd-95531636-1679437339
</code></pre>

<p>And the challenge:</p>

<pre><code>Name:         letsencrypt-tls-cm-1259919220-2936945618-694921812
Namespace:    myApp-dev
Labels:       &lt;none&gt;
Annotations:  &lt;none&gt;
API Version:  acme.cert-manager.io/v1alpha3
Kind:         Challenge
Metadata:
  Creation Timestamp:  2020-06-15T11:59:55Z
  Finalizers:
    finalizer.acme.cert-manager.io
  Generation:  1
  Owner References:
    API Version:           acme.cert-manager.io/v1alpha2
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Order
    Name:                  letsencrypt-tls-cm-1259919220-2936945618
    UID:                   4d8eab8e-449b-494e-a751-912a77671223
  Resource Version:        1218492
  Self Link:               /apis/acme.cert-manager.io/v1alpha3/namespaces/myApp-dev/challenges/letsencrypt-tls-cm-1259919220-2936945618-694921812
  UID:                     8b355336-309a-4192-83b7-41397ebc20ac
Spec:
  Authz URL:  https://acme-v02.api.letsencrypt.org/acme/authz-v3/5253543313
  Dns Name:   cm-myApp-dev.dev
  Issuer Ref:
    Group:  cert-manager.io
    Kind:   Issuer
    Name:   letsencrypt-prod
  Key:      0USdpDsQg7_NY1FB138oj6O3AtVVKn6rkdxUSBQk4KI.qZ3FGlVmwRY6MwBNqUR5iktM1fJWdXxFWZYFOpjSUkQ
  Solver:
    http01:
      Ingress:
        Class:  nginx
        Pod Template:
          Metadata:
          Spec:
            Node Selector:
              kubernetes.io/os:  linux
  Token:                         0USdpDsQg7_NY1FB138oj6O3AtVVKn6rkdxUSBQk4KI
  Type:                          http-01
  URL:                           https://acme-v02.api.letsencrypt.org/acme/chall-v3/5253543313/1eUG0g
  Wildcard:                      false
Status:
  Presented:   true
  Processing:  true
  Reason:      Waiting for http-01 challenge propagation: failed to perform self check GET request 'http://cm-myApp-dev.dev/.well-known/acme-challenge/0USdpDsQg7_NY1FB138oj6O3AtVVKn6rkdxUSBQk4KI': Get "http://cm-myApp-dev.dev/.well-known/acme-challenge/0USdpDsQg7_NY1FB138oj6O3AtVVKn6rkdxUSBQk4KI": dial tcp: lookup cm-myApp-dev.dev on 10.0.0.10:53: no such host
  State:       pending
Events:
  Type    Reason     Age    From          Message
  ----    ------     ----   ----          -------
  Normal  Started    2m15s  cert-manager  Challenge scheduled for processing
  Normal  Presented  2m14s  cert-manager  Presented challenge using http-01 challenge mechanism
</code></pre>

<p>I'm quite new to kubernetes and don't know where to look to fix the error bellow, any help is greatly appreciated.</p>

<pre><code>Waiting for http-01 challenge propagation: failed to perform self check GET request 'http://cm-myApp-dev.dev/.well-known/acme-challenge/0USdpDsQg7_NY1FB138oj6O3AtVVKn6rkdxUSBQk4KI': Get "http://cm-myApp-dev.dev/.well-known/acme-challenge/0USdpDsQg7_NY1FB138oj6O3AtVVKn6rkdxUSBQk4KI": dial tcp: lookup cm-myApp-dev.dev on 10.0.0.10:53: no such host
</code></pre>

<p>Looking in the ingress controller i get the following error:</p>

<pre><code>    7 controller.go:1374] Error getting SSL certificate "myApp-dev/letsencrypt-tls-cd": local SSL certificate myApp-dev/letsencrypt-tls-cd was not found
W0616 06:24:29.033235       7 controller.go:1119] Error getting SSL certificate "myApp-dev/letsencrypt-tls-cm": local SSL certificate myApp-dev/letsencrypt-tls-cm was not found. Using default certificate
W0616 06:24:29.033264       7 controller.go:1374] Error getting SSL certificate "myApp-dev/letsencrypt-tls-cd": local SSL certificate myApp-dev/letsencrypt-tls-cd was not found
I0616 06:24:50.355937       7 status.go:275] updating Ingress myApp-dev/cm-acme-http-solver-9z88h status from [] to [{10.240.0.252 } {10.240.1.58 }]
W0616 06:24:50.363181       7 controller.go:1119] Error getting SSL certificate "myApp-dev/letsencrypt-tls-cm": local SSL certificate myApp-dev/letsencrypt-tls-cm was not found. Using default certificate
W0616 06:24:50.363346       7 controller.go:1374] Error getting SSL certificate "myApp-dev/letsencrypt-tls-cd": local SSL certificate myApp-dev/letsencrypt-tls-cd was not found
I0616 06:24:50.363514       7 event.go:278] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"myApp-dev", Name:"cm-acme-http-solver-9z88h", UID:"1b53f4dc-1b52-4f11-9cd0-6ffe1d0d9d40", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"1451371", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress myApp-dev/cm-acme-http-solver-9z88h
</code></pre>

<p>If someone Googles this, then know that this issue can be also caused by DNS caching in your Kubernetes cluster. In this case, it is a transient error, but in some contexts speed could be important (e.g. if you are a managed service provider).</p>
<p>I <a href="https://www.usepine.com/blog/en/improving-cert-manager-self-check-speed-when-issuing-certificates/" rel="noreferrer">wrote about it here</a> but in summary.</p>
<ul>
<li>cert-manager would emit the &quot;no such host&quot; error for a while, and eventually succeed</li>
<li>my coredns ConfigMap (in <code>kube-system</code> namespace) stipulated local DNS resolvers, and a 30 sec cache</li>
<li>you can fix the delay by <em>(1) removing the cache</em>, and <em>(2) pointing the resolver to Google DNS</em> (or another, depending on your needs)</li>
</ul>
<p>Hope this pointer is helpful to someone.</p>
