<p>I'm setting up a Kubernetes cluster in AWS using Kops. I've got an nginx ingress controller, and I'm trying to use letsencrypt to setup tls. Right now I can't get my ingress up and running because my certificate challenges get this error:</p>
<p><code>Waiting for http-01 challenge propagation: failed to perform self check GET request 'http://critsit.io/.well-known/acme-challenge/[challengeId]': Get http://critsit.io/.well-known/acme-challenge/[challengeId]: EOF</code></p>
<p>I've got a LoadBalancer service that's taking public traffic, and the certificate issuer automatically creates 2 other services which don't have public IPs.</p>
<p>What am I doing wrong here? Is there some networking issue preventing the pods from finishing the acme flow? Or maybe something else?</p>
<p>Note: I have setup an A record in Route53 to direct traffic to the LoadBalancer.</p>
<pre><code>&gt; kubectl get services
NAME                        TYPE           CLUSTER-IP       EXTERNAL-IP                                                               PORT(S)                      AGE
cm-acme-http-solver-m2q2g   NodePort       100.69.86.241    &lt;none&gt;                                                                    8089:31574/TCP               3m34s
cm-acme-http-solver-zs2sd   NodePort       100.67.15.149    &lt;none&gt;                                                                    8089:30710/TCP               3m34s
default-http-backend        NodePort       100.65.125.130   &lt;none&gt;                                                                    80:32485/TCP                 19h
kubernetes                  ClusterIP      100.64.0.1       &lt;none&gt;                                                                    443/TCP                      19h
landing                     ClusterIP      100.68.115.188   &lt;none&gt;                                                                    3001/TCP                     93m
nginx-ingress               LoadBalancer   100.67.204.166   [myELB].us-east-1.elb.amazonaws.com                                       443:30596/TCP,80:30877/TCP   19h
</code></pre>
<p>Here's my ingress setup:</p>
<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: critsit-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: &quot;nginx&quot;
    cert-manager.io/acme-challenge-type: &quot;http01&quot;
    cert-manager.io/cluster-issuer: &quot;letsencrypt-prod&quot;
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  tls:
    - hosts:
      - critsit.io
      - app.critsit.io
      secretName: letsencrypt-prod
  rules:
    - host: critsit.io
      http:
        paths:
          - path: /
            backend:
              serviceName: landing
              servicePort: 3001
</code></pre>
<p>And my certificate issuer:</p>
<pre><code>apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: michael.vegeto@gmail.com
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class:  nginx
      selector: {}
</code></pre>
<p>Update: I've noticed that my load balancer has all of the instances marked as OutOfOrder because they're failing health checks. I wonder if that's related to the issue.</p>
<p>Second update: I abandoned this route altogether, and rebuilt my networking/ingress system using Istio</p>

<p>The error message you are getting can mean a wide variety of issues. However, there are few things you can check/do in order to make it work:</p>
<ol>
<li>Delete the Ingress, the certificates and the cert-manager <a href="https://cert-manager.io/docs/installation/uninstall/kubernetes/" rel="nofollow noreferrer">fully</a>. After that add them all back to make sure it installs clean. Sometimes stale certs or bad/multi Ingress pathing might be the issue. For example you can use Helm:</li>
</ol>
<hr />
<pre><code>helm install my-nginx-ingress stable/nginx-ingress
helm repo add jetstack https://charts.jetstack.io
helm repo update 
helm install  cert-manager jetstack/cert-manager  --namespace cert-manager --version v0.15.0 --set installCRDs=true
</code></pre>
<hr />
<ol start="2">
<li><p>Make sure your traffic allows HTTP or has HTTPS with a trusted cert.</p>
</li>
<li><p>Check if hairpin mode of your loadbalancer and make sure it is working.</p>
</li>
<li><p>Add: <code>nginx.ingress.kubernetes.io/ssl-redirect: &quot;false&quot;</code> annotation to the Ingress rule. Wait a moment and see if valid cert will be created.</p>
</li>
<li><p>You can manually manually issue certificates in your Kubernetes cluster. To do so, please follow <a href="https://github.com/nabsul/k8s-letsencrypt" rel="nofollow noreferrer">this guide</a>.</p>
</li>
<li><p>The problem can solve itself in time. Currently if the self check fails, it
updates the status information with the reason (like: self check failed) and than
tries again later (to allow for propagation). This is an expected behavior.</p>
</li>
</ol>
<p>This is an ongoing issue that is being tracked <a href="https://github.com/jetstack/cert-manager/issues/656" rel="nofollow noreferrer">here</a> and <a href="https://github.com/jetstack/cert-manager/issues/3238" rel="nofollow noreferrer">here</a>.</p>
