<p>I am using Promethues to monitor my Kubernetes cluster. All my microservices can be accessed using my HA Proxy.</p>

<p>My base Promethues config is :</p>

<pre><code>- job_name: 'kubernetes_pods'
  tls_config:
    insecure_skip_verify: true
  kubernetes_sd_configs:
  - api_server: http://172.29.219.102:8080
    role: pod
  relabel_configs:
    - source_labels: [__meta_kubernetes_pod_host_ip]
      target_label: __address__
      regex: (.*)
      replacement: 172.29.219.110:8080
</code></pre>

<p>Where <code>172.29.219.110:8080</code> is the IP &amp; Port of my standalone HA Proxy.</p>

<p>The endpoint that I am trying to monitor using Prometheus is <code>/auth/health</code>.</p>

<p>When I do a simple curl command from anywhere, I see :</p>

<pre><code># curl http://172.29.219.110:8080/auth/health
{"status":"UP"}
</code></pre>

<p>But when Prometheus tries to do it, the logs indicate :</p>

<pre><code>level=warn ts=2017-12-15T16:40:48.301741927Z caller=scrape.go:673 component="target manager" scrape_pool=kubernetes_pods target=http://172.29.219.110:8080/auth/health msg="append failed" err="no token found"
</code></pre>

<p>This endpoint is publicly exposed and requires no authentication what so ever. So why does Promethues say :</p>

<p><a href="https://i.stack.imgur.com/cLdQP.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/cLdQP.jpg" alt="enter image description here"></a></p>

<blockquote>
  <p>{"status":"UP"}</p>
</blockquote>

<p>Prometheus requires data to be in its format, and cannot handle other arbitrary data. The error you are getting is a parse error due to this.</p>

<p>You should instrument your code using a <a href="https://prometheus.io/docs/instrumenting/clientlibs/" rel="nofollow noreferrer">client library</a>, and have it expose data in the Prometheus text format.</p>
