<p>I'm using <code>mongodb-exporter</code> for store/query the metrics via <code>prometheus</code>. I have set up a custom metric server and storing values for that . </p>

<p>That is the evidence of <code>prometheus-exporter</code> and <code>custom-metric-server</code> works compatible .</p>

<p><strong>Query:</strong></p>

<pre><code>kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/monitoring/pods/*/mongodb_mongod_wiredtiger_cache_bytes"
</code></pre>

<p><strong>Result:</strong></p>

<pre><code>{"kind":"MetricValueList","apiVersion":"custom.metrics.k8s.io/v1beta1","metadata":{"selfLink":"/apis/custom.metrics.k8s.io/v1beta1/namespaces/monitoring/pods/%2A/mongodb_mongod_wiredtiger_cache_bytes"},"items":[{"describedObject":{"kind":"Pod","namespace":"monitoring","name":"mongo-exporter-2-prometheus-mongodb-exporter-68f95fd65d-dvptr","apiVersion":"/v1"},"metricName":"mongodb_mongod_wiredtiger_cache_bytes","timestamp":"TTTTT","value":"0"}]}
</code></pre>

<p>In my case when I create a hpa for this custom metrics from mongo exporter, hpa return this error to me :</p>

<pre><code>failed to get mongodb_mongod_wiredtiger_cache_bytes utilization: unable to get metrics for resource mongodb_mongod_wiredtiger_cache_bytes: no metrics returned from resource metrics API
</code></pre>

<p>What is the main issue on my case ? I have checked all configs and flow is looking fine, but where is the my mistake .</p>

<p>Help </p>

<p>Thanks :) </p>

Ayhan Balik
<p>In comments you wrote that you have enabled <code>external.metrics</code>, however in original question you had issues with <code>custom.metrics</code></p>

<p>In short:</p>

<ul>
<li><code>metrics</code> supports only basic metric like CPU or Memory.</li>
<li><code>custom.metrics</code> allows you to extend basic metrics to all Kubernetes objects (http_requests, number of pods, etc.).</li>
<li><code>external.metrics</code> allows to gather metrics which are not Kubernetes objects:</li>
</ul>

<blockquote>
  <p>External metrics allow you to autoscale your cluster based on any
  metric available in your monitoring system. Just provide a metric
  block with a name and selector, as above, and use the External metric
  type instead of Object</p>
</blockquote>

<p>For more detailed description, please check <a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-metrics-not-related-to-kubernetes-objects" rel="nofollow noreferrer">this doc</a>.</p>

<p><strong>Minikube</strong></p>

<p>To verify if <code>custom.metrics</code> are enabled you need to execute command below and check if you can see any <code>metrics-server...</code> pod.</p>

<pre><code>$ kubectl get pods -n kube-system
...
metrics-server-587f876775-9qrtc    1/1     Running   4          5d1h
</code></pre>

<p>Second way is to check if <code>minikube</code> have enabled <code>metrics-server</code> by</p>

<pre><code>$ minikube addons list
...
- metrics-server: enabled
</code></pre>

<p>If it is disabled just execute </p>

<pre><code>$ sudo minikube addons enable metrics-server
✅  metrics-server was successfully enabled
</code></pre>

<p><strong>GKE</strong></p>

<p>Currently at GKE <code>heapster</code> and <code>metrics-server</code> are turn on as default but <code>custom.metrics</code> are not supported by default.
You have to install <code>prometheus adapter</code> or <code>stackdriver</code>.</p>

<p><strong>Kubeadm</strong></p>

<p>Kubeadm do not include <code>heapster</code> or <code>metrics server</code> at the beginning. For easy installation, you can use <a href="https://github.com/luxas/kubeadm-workshop/blob/master/demos/monitoring/metrics-server.yaml" rel="nofollow noreferrer">this YAML</a>.</p>

<p>Later you have to install <code>prometheus adapter</code>.</p>

<p><strong>Apply custom.metrics</strong></p>

<p>It's the same for <code>Minikube</code>, <code>Kubeadm</code>, <code>GKE</code>.</p>

<p>Easiest way to apply <code>custom.metrics</code> is to install <a href="https://github.com/DirectXMan12/k8s-prometheus-adapter" rel="nofollow noreferrer">prometheus adapter</a> via <code>Helm</code>. </p>

<p>After helm installation you will be able to see note:</p>

<pre><code>NOTES:
my-release-prometheus-adapter has been deployed.
In a few minutes you should be able to list metrics using the following command(s):

  kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1
</code></pre>

<p>As additional information, you can use <code>jq</code> to get more user friendly output.</p>

<p><code>kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq .</code></p>
