<p>I have a K8s cluster with colocated etcd deployed on-prem servers, using Kubespray. I don't see the etcd metrics getting scraped by Prometheus operator. Prometheus operator deployed using helm <code>v3.5.4</code>.
K8s version <code>1.22</code> , Helm chart <code>prometheus-community/kube-prometheus-stack</code> version <code>25.0.0</code> , 3 node control plane on <code>CentOS 7</code>.</p>
<p>Prometheus config shows a job for etcd <code>- job_name: serviceMonitor/monitoring/kube-prometheus-kube-prome-kube-etcd/0</code> .
But there is no service for etcd in the list of Services for Prometheus.</p>
<p>There are no endpoints defined for etcd</p>
<p><a href="https://i.stack.imgur.com/oWbN5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/oWbN5.png" alt="Missing endpoints for etcd" /></a></p>
<p>Values.yml (updated with volumes ) for helm deployment</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>prometheus:
    service: 
        type: NodePort
        externalTrafficPolicy: Local
    ingress:
        enabled: true
        annotations:
            kubernetes.io/ingress.class: "custom"
        hosts:
            - prometheus.{{ cluster_domain }}.mydomain.com
        paths: 
            - /
        pathType: Prefix
        tls:
            - secretName: 
    prometheusSpec:
        storageSpec:
            volumeClaimTemplate:
                spec:
                    accessModes: ["ReadWriteOnce"]
                    storageClassName: rook-ceph-block
                    resources:
                        requests:
                            storage: {{ monitoring.storage_size }}
        volumeMounts:
        - name: cert-vol
          mountPath: "/etc/prometheus/secrets/etcd-certs"
          readOnly: true

        volumes:
        - name: cert-vol
          secret: 
            secretName: etcd-certs

kubeEtcd:
    enabled: true
    endpoints:
      - 172.1.1.1
      - 172.1.1.2
      - 172.1.1.3
  
    service:
        port: 2379
        targetPort: 2379
    serviceMonitor:
        scheme: https
        insecureSkipVerify: true
        caFile: /etc/prometheus/secrets/etcd-certs/ca.crt
        certFile: /etc/prometheus/secrets/etcd-certs/client.crt
        keyFile: /etc/prometheus/secrets/etcd-certs/client.key</code></pre>
</div>
</div>
</p>

RDs
<p>I added the endpoints to kubeEtcd section to get it to work. The updated values.yaml is like below (changed IP adresses):</p>
<pre><code>prometheus:
    service: 
        type: NodePort
        externalTrafficPolicy: Local
    ingress:
        enabled: true
        annotations:
            kubernetes.io/ingress.class: &quot;custom&quot;
        hosts:
            - prometheus.{{ cluster_domain }}.mydomain.com
        paths: 
            - /
        pathType: Prefix
        tls:
            - secretName: 
    prometheusSpec:
        storageSpec:
            volumeClaimTemplate:
                spec:
                    accessModes: [&quot;ReadWriteOnce&quot;]
                    storageClassName: rook-ceph-block
                    resources:
                        requests:
                            storage: {{ monitoring.storage_size }}
        volumeMounts:
        - name: cert-vol
          mountPath: &quot;/etc/prometheus/secrets/etcd-certs&quot;
          readOnly: true

        volumes:
        - name: cert-vol
          secret: 
            secretName: etcd-certs

kubeEtcd:
    enabled: true
    endpoints:
      - 172.1.1.1
      - 172.1.1.2
      - 172.1.1.3
  
    service:
        port: 2379
        targetPort: 2379
    serviceMonitor:
        scheme: https
        insecureSkipVerify: true
        caFile: /etc/prometheus/secrets/etcd-certs/ca.crt
        certFile: /etc/prometheus/secrets/etcd-certs/client.crt
        keyFile: /etc/prometheus/secrets/etcd-certs/client.key
</code></pre>
