<p>I've been struggling for a while trying to get HTTPS access to my Elasticsearch cluster in Kubernetes.</p>

<p>I <em>think</em> the problem is that Kubernetes doesn't like the TLS certificate I'm trying to use, which is why it's not passing it all the way through to the browser.</p>

<p>Everything else seems to work, since when I accept the Kubernetes Ingress Controller Fake Certificate, the requests go through as expected.</p>

<p>In my attempt to do this I've set up:</p>

<ul>
<li>The cluster itself</li>
<li>An nginx-ingress controller</li>
<li>An ingress resource</li>
</ul>

<p>Here's the related yaml:</p>

<ul>
<li><p>Cluster:</p>

<pre><code>apiVersion: v1
kind: Service
metadata:
    creationTimestamp: 2018-08-03T03:20:47Z
labels:
    run: my-es
name: my-es
namespace: default
resourceVersion: "3159488"
selfLink: /api/v1/namespaces/default/services/my-es
uid: 373047e0-96cc-11e8-932b-42010a800043
spec:
    clusterIP: 10.63.241.39
ports:
    - name: http
port: 8080
protocol: TCP
targetPort: 9200
selector:
    run: my-es
sessionAffinity: None
type: ClusterIP
status:
    loadBalancer: {}
</code></pre></li>
<li><p>The ingress resource</p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    annotations:kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS
nginx.ingress.kubernetes.io/cors-origins: http://localhost:3425 https://mydomain.ca
    https://myOtherDomain.ca
        nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
creationTimestamp: 2018-08-12T08:44:29Z
generation: 16
name: es-ingress
namespace: default
resourceVersion: "3159625"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/es-ingress
uid: ece0071d-9e0b-11e8-8a45-42001a8000fc
spec:
    rules:
        - http:
paths:
    - backend:
serviceName: my-es
servicePort: 8080
path: /
tls:
    - hosts:
- mydomain.ca
secretName: my-tls-secret
status:
    loadBalancer:
        ingress:
            - ip: 130.211.179.225
</code></pre></li>
<li><p>The nginx-ingress controller:</p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2018-08-12T00:41:32Z
  labels:
    app: nginx-ingress
    chart: nginx-ingress-0.23.0
    component: controller
    heritage: Tiller
    release: nginx-ingress
  name: nginx-ingress-controller
  namespace: default
  resourceVersion: "2781955"
  selfLink: /api/v1/namespaces/default/services/nginx-ingress-controller
  uid: 755ee4b8-9dc8-11e8-85a4-4201a08000fc
spec:
  clusterIP: 10.63.250.256
  externalTrafficPolicy: Cluster
  ports:
  - name: http
    nodePort: 32084
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 31182
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app: nginx-ingress
    component: controller
    release: nginx-ingress
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 35.212.6.131
</code></pre></li>
</ul>

<p>I feel like I'm missing something basic, because it doesn't seem like it should be this hard to expose something this simple...</p>

<p>To get my certificate, I just requested one for mydomain.ca from godaddy.</p>

<p>Do I need to somehow get a certificate using my ingress resource's cluster IP as the common name? </p>

<p>It doesn't seem possible to verify ownership of an IP.</p>

<p>I've seen people mention ways for Kubernetes to automatically create certificates for ingress resources, but those seem to be self signed.</p>

<p>Here are some logs from the nginx-controller:</p>

<p>This one is talking about a PEM with the tls-secret, but it's only a warning.</p>

<pre><code>{
 insertId:  "1kvvhm7g1q7e0ej"
 labels: {
  compute.googleapis.com/resource_name:  "fluentd-gcp-v2.0.17-5b82n"
  container.googleapis.com/namespace_name:  "default"
  container.googleapis.com/pod_name:  "nginx-ingress-controller-58f57fc597-zl25s"
  container.googleapis.com/stream:  "stderr"
 }
 logName:  "projects/project-7d320/logs/nginx-ingress-controller"
 receiveTimestamp:  "2018-08-14T02:58:42.135388365Z"
 resource: {
  labels: {
   cluster_name:  "my-elasticsearch-cluster"
   container_name:  "nginx-ingress-controller"
   instance_id:  "2341889542400230234"
   namespace_id:  "default"
   pod_id:  "nginx-ingress-controller-58f57fc597-zl25s"
   project_id:  "project-7d320"
   zone:  "us-central1-a"
  }
  type:  "container"
 }
 severity:  "WARNING"
 textPayload:  "error obtaining PEM from secret default/my-tls-cert: error retrieving secret default/my-tls-cert: secret default/my-tls-cert was not found"
 timestamp:  "2018-08-14T02:58:37Z"
}
</code></pre>

<p>I have a few occurences of this handshake error, which may be a result of the last warning...</p>

<pre><code>{
 insertId:  "148t6rfg1xmz978"
 labels: {
  compute.googleapis.com/resource_name:  "fluentd-gcp-v2.0.17-5b82n"
  container.googleapis.com/namespace_name:  "default"
  container.googleapis.com/pod_name:  "nginx-ingress-controller-58f57fc597-zl25s"
  container.googleapis.com/stream:  "stderr"
 }
 logName:  "projects/project-7d320/logs/nginx-ingress-controller"
 receiveTimestamp:  "2018-08-14T15:55:52.438035706Z"
 resource: {
  labels: {
   cluster_name:  "my-elasticsearch-cluster"
   container_name:  "nginx-ingress-controller"
   instance_id:  "2341889542400230234"
   namespace_id:  "default"
   pod_id:  "nginx-ingress-controller-58f57fc597-zl25s"
   project_id:  "project-7d320"
   zone:  "us-central1-a"
  }
  type:  "container"
 }
 severity:  "ERROR"
 textPayload:  "2018/08/14 15:55:50 [crit] 1548#1548: *860 SSL_do_handshake() failed (SSL: error:1417D18C:SSL routines:tls_process_client_hello:version too low) while SSL handshaking, client: 127.0.0.1, server: 0.0.0.0:442"
 timestamp:  "2018-08-14T15:55:50Z"
}
</code></pre>

<p>The above logs make it seem like my tls secret isnt working, but when I run kubectl describe ingress, it says my secret terminates.</p>

<pre><code>aaronmw@project-7d320:~$ kubectl describe ing
Name:             es-ingress
Namespace:        default
Address:          130.221.179.212
Default backend:  default-http-backend:80 (10.61.3.7:8080)
TLS:
  my-tls-secret terminates mydomain.ca
Rules:
  Host  Path  Backends
  ----  ----  --------
  *
        /   my-es:8080 (&lt;none&gt;)
Annotations:
Events:  &lt;none&gt;
</code></pre>

<p>I figured it out!</p>

<p>What I ended up doing was adding a default ssl certificate to my nginx-ingress controller on creation using the following command</p>

<pre><code>helm install --name nginx-ingress --set controller.extraArgs.default-ssl-certificate=default/search-tls-secret stable/nginx-ingress
</code></pre>

<p>Once I had that, it was passing the cert as expected, but I still had the wrong cert as the CN didn't match my load balancer IP.</p>

<p>So what I did was:</p>

<ul>
<li>Make my load balancer IP static</li>
<li>Add an A record to my domain, to map a subdomain to that IP</li>
<li>Re-key my cert to match that new subdomain</li>
</ul>

<p>And I'm in business!</p>

<p>Thanks to @Crou, who's comment reminded me to look at the logs and got me on the right track.</p>
