<p>I am new to kubernetes and I finally realized how to launch the metrics-server as documented <a href="https://github.com/kubernetes-sigs/metrics-server" rel="nofollow noreferrer">kubernetes-sigs/metrics-server</a>. In case that someone else wonders you need to deploy on Master node and also have minimum one worker in the cluster.</p>
<p>So I get this error:</p>
<pre class="lang-sh prettyprint-override"><code>E0818 15:25:22.835094       1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:&lt;hostname-master&gt;: unable to fetch metrics from Kubelet &lt;hostname-master&gt; (&lt;hostname-master&gt;): Get https://&lt;hostname-master&gt;:10250/stats/summary?only_cpu_and_memory=true: x509: certificate signed by unknown authority, unable to fully scrape metrics from source kubelet_summary:&lt;hostname-worker&gt;: unable to fetch metrics from Kubelet &lt;hostname-worker&gt; (&lt;hostname-worker&gt;): Get https://&lt;hostname-worker&gt;:10250/stats/summary?only_cpu_and_memory=true: x509: certificate signed by unknown authority]
</code></pre>
<p>I am using my own CAs (not self signed) and I have modified the components.yml file (sample):</p>
<pre class="lang-sh prettyprint-override"><code>args:
  - --cert-dir=/tmp/metricsServerCas
  - --secure-port=4443
  - --kubelet-preferred-address-types=Hostname
</code></pre>
<p>I know that I can disable the tls by using this flag <code>--kubelet-insecure-tls</code> I have already tried it. I want to use my own CAs for extra security.</p>
<p>I have see other many relevant questions (few samples) e.g.:</p>
<p><a href="https://stackoverflow.com/questions/36939381/x509-certificate-signed-by-unknown-authority-kubernetes">x509 certificate signed by unknown authority- Kubernetes</a> and <a href="https://stackoverflow.com/questions/46234295/kubectl-unable-to-connect-to-server-x509-certificate-signed-by-unknown-authori">kubectl unable to connect to server: x509: certificate signed by unknown authority</a></p>
<p>Although that I have applied chown already my <code>$HOME/.kube/config</code> still I see this error.</p>
<p>Where am I going wrong?</p>
<p><strong>Update:</strong> On the worker I am creating a directory e.g. <code>/tmp/ca</code> and I add the ca file(s) in the directory.</p>
<p>I am not really good yet with the mounting points and I assume that I am doing something wrong. The default syntax of the images can be found here <a href="https://github.com/kubernetes-sigs/metrics-server/releases/tag/v0.3.7" rel="nofollow noreferrer">kubernetes-sigs/metrics-server/v0.3.7</a> (see components.yml file).</p>
<p>I tried to create a directory on my worker e.g. /tmp/ca and I modified the flag <code>--cert-dir=/tmp/ca</code> and <code>mountPath: /tmp/ca</code></p>
<p>When I am deploying the file e.g.:</p>
<pre class="lang-sh prettyprint-override"><code>kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml
</code></pre>
<p>I keep getting the error from the metrics-server-xxxx:</p>
<pre class="lang-sh prettyprint-override"><code>panic: open /tmp/client-ca-file805316981: read-only file system
</code></pre>
<p>Although I have given full access to the directory e.g.:</p>
<pre class="lang-sh prettyprint-override"><code>$ ls -la /tmp/ca
total 8
drwxr-xr-x.  2 user user   20 Aug 19 16:59 .
drwxrwxrwt. 18 root        root        4096 Aug 19 17:34 ..
-rwxr-xr-x.  1 user user 1025 Aug 19 16:59 ca.crt
</code></pre>
<p>I am not sure where I am going wrong.</p>
<p>How is meant to be configured so someone can use non self signed certificates? I can see that most people are using non SSL which I would like to avoid.</p>
<p>Sample of my args in the image:</p>
<pre class="lang-sh prettyprint-override"><code>spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  template:
    metadata:
      name: metrics-server
      labels:
        k8s-app: metrics-server
    spec:
      serviceAccountName: metrics-server
      volumes:
      # mount in tmp so we can safely use from-scratch images and/or read-only containers
      - name: tmp-dir
        emptyDir: {}
      containers:
      - name: metrics-server
        image: k8s.gcr.io/metrics-server/metrics-server:v0.3.7
        imagePullPolicy: IfNotPresent
        args:
          - --cert-dir=/tmp/ca
          - --secure-port=4443
          - --kubelet-preferred-address-types=Hostname
        ports:
        - name: main-port
          containerPort: 4443
          protocol: TCP
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - name: tmp-dir
          mountPath: /tmp/ca
      nodeSelector:
        kubernetes.io/os: linux
        kubernetes.io/arch: &quot;amd64&quot;
</code></pre>
<p><strong>Update 2:</strong> Adding curl command from Master to Worker including error output:</p>
<pre class="lang-sh prettyprint-override"><code>$ curl --cacert /etc/kubernetes/pki/ca.crt https://node_hostname:10250/stats/summary?only_cpu_and_memory=true
curl: (60) Peer's certificate issuer has been marked as not trusted by the user.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a &quot;bundle&quot;
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
</code></pre>

Thanos
<p>Posting this answer as a community wiki to give better visibility as the solution was posted in the comments.</p>
<blockquote>
<p>The version that I used before was 1.18.2 and metrics server v0.3.6. Deployment was through kubeadm. Yes all requirements was exactly as the metrics-server/requirements. The good news is that I got it running by upgrading my k8s version on 1.19.0 and using the latest version v0.3.7. It works with self signed certificates.</p>
</blockquote>
<p>The issue was resolved by upgrading:</p>
<ul>
<li><code>Kubernetes</code>: <code>1.18.2</code> -&gt; <code>1.19.0</code></li>
<li><code>Metrics-server</code>: <code>0.3.6</code> -&gt; <code>0.3.7</code></li>
</ul>
<p>This upgrade allowed to run <code>metrics-server</code> with <code>tls</code> enabled (self-signed certificates).</p>
<hr />
<p>Additional resources that could help when deploying <code>metrics-server</code> with <code>tls</code>:</p>
<ul>
<li><em><a href="https://github.com/kubernetes-sigs/metrics-server/blob/master/FAQ.md#how-to-run-metrics-server-securely" rel="nofollow noreferrer">Github.com: Kubernetes-sigs: Metrics-server: FAQ: How to run metrics-server-securely</a></em></li>
</ul>
<blockquote>
<p>How to run metrics-server securely?
Suggested configuration:</p>
<ul>
<li>Cluster with <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/" rel="nofollow noreferrer">RBAC</a> enabled</li>
<li>Kubelet <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/#options" rel="nofollow noreferrer">read-only port</a> port disabled</li>
<li>Validate kubelet certificate by mounting CA file and providing --kubelet-certificate-authority flag to metrics server</li>
<li>Avoid passing insecure flags to metrics server (--deprecated-kubelet-completely-insecure, --kubelet-insecure-tls)</li>
<li>Consider using your own certificates (--tls-cert-file, --tls-private-key-file)</li>
</ul>
</blockquote>
<ul>
<li><em><a href="https://github.com/kubernetes-sigs/metrics-server/issues/146" rel="nofollow noreferrer">Github.com: Metrics-server: x509: certificate signed by unknown authority</a></em></li>
<li><em><a href="https://ftclausen.github.io/general/setting_up_k8s_with_metrics_server/" rel="nofollow noreferrer">Ftclausen.github.io: Setting up K8S with metrics-server</a></em></li>
</ul>
