<p><strong>Problem setting.</strong> Suppose I have 2 pods, A and B. I want to be able to dynamically scale pod A based on some arbitrary number from some arbitrary source. Suppose that pod B is such a source: for example, it can have an HTTP server with an endpoint which responds with the number of desired replicas of pod A at the moment of request. Or maybe it is an ES server or a SQL DB (does not matter).</p>
<p><strong>Question.</strong> What kubernetes objects do I need to define to achieve this (apart from HPA)? What configuration should HPA have to know that it needs to look up B for current metric? How should API of B look like (or is there any constraints?)?</p>
<p><strong>Research I have made.</strong> Unfortunately, the official documentation does not say much about it, apart from declaring that there is such a possibility. There are also two repositories, <a href="https://github.com/kubernetes-sigs/custom-metrics-apiserver" rel="noreferrer">one</a> with some go boilerplate code that I have trouble building and <a href="https://github.com/zalando-incubator/kube-metrics-adapter" rel="noreferrer">another</a> one that has no usage instructions whatsoever (though allegedly does fulfil the &quot;external metrics over HTTP&quot; requirement).</p>
<p>By having a look at the <code>.yaml</code> configs in those repositories, I have reached a conclusion that apart from <code>Deployment</code> and <code>Service</code> one needs to define an <code>APIService</code> object that registers the external or custom metric in the kubernetes API and links it with a normal service (where you would have your pod) and a handful of <code>ClusterRole</code> and <code>ClusterRoleBinding</code> objects. But there is no explanation about it. Also I could not even list existing APIServices with kubectl in my local cluster (of 1.15 version) like other objects.</p>

<p>The easiest way will be to feed metrics into Prometheus (which is a commonly solved problem), and then setup a Prometheus-based HPA (also a commonly solved problem).</p>
<p><strong>1. Feed own metrics to Prometheus</strong></p>
<ul>
<li>Start with <a href="https://github.com/helm/charts/tree/master/stable/prometheus-operator" rel="noreferrer">Prometheus-Operator</a> to get the cluster itself monitored, and get access to <code>ServiceMonitor</code> objects. ServiceMonitors are pointers to services in the cluster. They let your pod's <code>/metrics</code> endpoint be discovered and scraped by a prometheus server.</li>
<li>Write a pod that reads metrics from your 3rd party API and shows them in own <code>/metrics</code> endpoint. This will be the adapter between your API and Prometheus format. There are clients of course: <a href="https://github.com/prometheus/client_python#exporting" rel="noreferrer">https://github.com/prometheus/client_python#exporting</a></li>
<li>Write a <code>Service</code> of type <code>ClusterIP</code> that represents your pod.</li>
<li>Write a <code>ServiceMonitor</code> that points to a service.</li>
<li>Query your custom metrics thru Prometheus dashboard to ensure this stage is done.</li>
</ul>
<p><strong>2. Setup Prometheus-based HPA</strong></p>
<ul>
<li>Setup <a href="https://github.com/helm/charts/tree/master/stable/prometheus-adapter" rel="noreferrer">Prometheus-Adapter</a> and follow the HPA walkthrough.</li>
<li>Or follow the guide <a href="https://github.com/stefanprodan/k8s-prom-hpa" rel="noreferrer">https://github.com/stefanprodan/k8s-prom-hpa</a></li>
</ul>
<p>This looks like a huge pile of work to get the HPA. However, only the adapter pod is a custom part here. Everything else is a standard stack setup in most of the clusters, and you will get many other use cases for it anyways.</p>
