<p>I'm trying to set up a K3s cluster. When I had a single master and agent setup cert-manager had no issues. Now I'm trying a 2 master setup with embedded etcd. I opened TCP ports <code>6443</code> and <code>2379-2380</code> for both VMs and did the following:</p>
<pre class="lang-none prettyprint-override"><code>VM1: curl -sfL https://get.k3s.io | sh -s server --token TOKEN --cluster-init
VM2: curl -sfL https://get.k3s.io | sh -s server --token TOKEN --server https://MASTER_IP:6443
</code></pre>
<pre class="lang-none prettyprint-override"><code># k3s kubectl get nodes
NAME  STATUS   ROLES                       AGE    VERSION
VM1   Ready    control-plane,etcd,master   130m   v1.22.7+k3s1
VM2   Ready    control-plane,etcd,master   128m   v1.22.7+k3s1
</code></pre>
<p>Installing cert-manager works fine:</p>
<pre class="lang-none prettyprint-override"><code># k3s kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.0/cert-manager.yaml
# k3s kubectl get pods --namespace cert-manager
NAME                                       READY   STATUS
cert-manager-b4d6fd99b-c6fpc               1/1     Running
cert-manager-cainjector-74bfccdfdf-gtmrd   1/1     Running
cert-manager-webhook-65b766b5f8-brb76      1/1     Running
</code></pre>
<p>My manifest has the following definition:</p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: info@example.org
    privateKeySecretRef:
      name: letsencrypt-account-key
    solvers:
    - selector: {}
      http01:
        ingress: {}
</code></pre>
<p>Which results in the following error:</p>
<pre class="lang-none prettyprint-override"><code># k3s kubectl apply -f manifest.yaml
Error from server (InternalError): error when creating &quot;manifest.yaml&quot;: Internal error occurred: failed calling webhook &quot;webhook.cert-manager.io&quot;: failed to call webhook: Post &quot;https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s&quot;: context deadline exceeded
</code></pre>
<p>I tried disabling both firewalls, waiting a day, reset and re-setup, but the error persists. Google hasn't been much help either. The little info I can find goes over my head for the most part and no tutorial seems to do any extra steps.</p>

<p>A good starting point for troubleshooting issues with the webhook can be found int the <a href="https://cert-manager.io/docs/concepts/webhook/#known-problems-and-solutions" rel="nofollow noreferrer">docs</a>, e.g. there is a section for <a href="https://cert-manager.io/docs/concepts/webhook/#known-problems-and-solutions" rel="nofollow noreferrer">problems on GKE private clusters</a>.</p>
<p>In my case, however, this didn't really solve the problem. For me the issue was that when I played around with <code>cert-manager</code> I happen to install and uninstall it multiple times. It turned out that just removing the namespace, e.g. <code>kubectl delete namespace cert-manager</code> didn't remove the webhooks and other non-obvious resources.</p>
<p>Following the official guide for <a href="https://cert-manager.io/v1.2-docs/installation/uninstall/kubernetes/" rel="nofollow noreferrer">uninstalling cert-manager</a> and applying the manifests again solved the issue.</p>
