<p>We have a setup, where Metricbeat is deployed as a DaemonSet on a Kubernetes cluster (specifically -- AWS EKS).</p>

<p>All seems to be functioning properly, but the <strong>kubelet</strong> connection.</p>

<p>To clarify, the following module:</p>

<pre><code>- module: kubernetes
  enabled: true
  metricsets:
    - state_pod
  period: 10s
  hosts: ["kube-state-metrics.system:8080"]
</code></pre>

<p>works properly (the events flow into logstash/elastic).</p>

<p>This module configuration, however, doesn't work in any variants of hosts value (<code>localhost</code>/<code>kubernetes.default</code>/whatever):</p>

<pre><code>- module: kubernetes
  period: 10s
  metricsets:
    - pod
  hosts: ["localhost:10255"]
  enabled: true
  add_metadata: true
  in_cluster: true
</code></pre>

<blockquote>
  <p>NOTE: using cluster IP instead of localhost (so that it goes to
  control plane) also works (although doesn't retrieve the needed
  information, of course).</p>
  
  <p>The configuration above was taken directly from the Metricbeat
  documentation and immediately struck me as odd -- how does localhost
  get translated (from within Metricbeat docker) to corresponding
  kubelet?</p>
</blockquote>

<p>The error is, as one would expect, in light of the above:</p>

<pre><code>error making http request: Get http://localhost:10255/stats/summary: 
dial tcp [::1]:10255: connect: cannot assign requested address
</code></pre>

<p>which indicates some sort of connectivity issue.</p>

<p>However, when SSH-ing to any node Metricbeat is deployed on, <code>http://localhost:10255/stats/summary</code> provides the correct output:</p>

<pre><code>{
  "node": {
   "nodeName": "...",
   "systemContainers": [
    {
     "name": "pods",
     "startTime": "2018-12-06T11:22:07Z",
     "cpu": {
      "time": "2018-12-23T06:54:06Z",
      ...
     },
     "memory": {
      "time": "2018-12-23T06:54:06Z",
      "availableBytes": 17882275840,
      ....
</code></pre>

<p>I must be missing something very obvious. Any suggestion would do.</p>

<p>NOTE: I cross-posted (and got no response for a couple of days) the same on <a href="https://discuss.elastic.co/t/metricbeat-kubernetes-module-cant-connect-to-kubelet/161939" rel="nofollow noreferrer">Elasticsearch Forums</a></p>

ZenMaster
<p>Inject the Pod's Node's IP via the <code>valueFrom</code> provider in the <code>env:</code> list:</p>

<pre><code>env:
- name: HOST_IP
  valueFrom:
    fieldRef: status.hostIP
</code></pre>

<p>and then update the metricbeat config file to use the host's IP:</p>

<pre><code>hosts: ["${HOST_IP}:10255"]
</code></pre>

<p>which metricbeat will resolve via its <a href="https://www.elastic.co/guide/en/beats/libbeat/6.5/config-file-format-env-vars.html#config-file-format-env-vars" rel="noreferrer">environment variable config injection</a></p>
