<p>I'm trying to get my head around how to get prometheus <a href="https://hub.helm.sh/charts/stable/prometheus" rel="nofollow noreferrer">https://hub.helm.sh/charts/stable/prometheus</a> collect etcd stats. I understand I need to set tls for it, but have a hard time to find good way to do it without manual additional ansible steps. Is there the way I can get etcd certs on worker node and mount them to prometheus pod?</p>

<p>Following the <a href="https://jpweber.io/blog/monitoring-external-etcd-cluster-with-prometheus-operator/" rel="nofollow noreferrer">Monitoring External Etcd Cluster With Prometheus Operator</a> you can easily configure Prometheus to scrape metrics from ETCD.</p>

<blockquote>
  <p>We can do all of that by creating certs as kubernetes secrets and
  adding a tlsConfig to our service monitor. Let me walk you through the
  whole process.</p>
</blockquote>

<p>The steps are:</p>

<p>1) Create etcd <code>service</code></p>

<p>2) Create/attach <code>endpoints</code> for etcd service</p>

<p>3) Create service monitor with appropriate tlsconfig. below example</p>

<pre><code>apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: etcd
  name: etcd
  namespace: kube-system
spec:
  endpoints:
  - interval: 30s
    port: metrics
    scheme: https
    tlsConfig:
      caFile: /etc/prometheus/secrets/kube-etcd-client-certs/etcd-client-ca.crt
      certFile: /etc/prometheus/secrets/kube-etcd-client-certs/etcd-client.crt
      keyFile: /etc/prometheus/secrets/kube-etcd-client-certs/etcd-client.key
      serverName: etcd-cluster
  jobLabel: k8s-app
  selector:
    matchLabels:
      k8s-app: etcd
</code></pre>

<p>4) Create Etcd Client Certificates</p>

<p>5) Create Kubernetes Secrets along with previously created certificate and key for prometheus and  etcd ca. This will allow prometheus to securely connect to etcd.
Example:</p>

<pre><code>kubectl -n monitoring create secret kube-etcd-client-certs --from-file=etcd-client-ca.crt=etcd-client.ca.crt --from-file=etcd-client.crt=etcd-client.crt --from-file=etcd-client.key=etcd-client.key
</code></pre>

<p>6) Update prometheus.yaml to include there names of the created secrets.</p>

<p>7) delploy etcd-service,servicemonitor and prometheus manifests to cluster</p>

<pre><code>kubectl apply -f etcd-service.yaml
kubectl apply -f etcd-serviceMon.yaml
kubectl apply -f prometheus-prometheus.yaml
</code></pre>

<p>Enjoy</p>
