<p>In <a href="https://github.com/kubernetes-client/java" rel="nofollow noreferrer">Kubernetes-client Java api</a>, I can get a count of available and total deployed pod instances for a given app using this way:</p>

<pre><code>ApiClient defaultClient = Configuration.getDefaultApiClient();
AppsV1beta1Api apiInstance = new AppsV1beta1Api();
...
try {
    AppsV1beta1DeploymentList result = apiInstance.listDeploymentForAllNamespaces(_continue, fieldSelector, includeUninitialized, labelSelector, limit, pretty, resourceVersion, timeoutSeconds, watch);
    foreach(ExtensionsV1beta1Deployment extensionsDeployment : result.getItems() ) {
        Map&lt;String, String&gt; labels = extensionsDeployment.getMetadata().getLabels();
        String appName = labels.getOrDefault("app", "");
        ExtensionsV1beta1DeploymentStatus status = extensionsDeployment.getStatus();
        int availablePods = status.getAvailableReplicas();
        int deployedPods = status.getReplicas();
        if ( availablePods != deployedPods) {
             // Generate an alert
        }
    }
} catch (ApiException e) {
    System.err.println("Exception when calling AppsV1beta1Api#listDeploymentForAllNamespaces");
    e.printStackTrace();
}
</code></pre>

<p>In above example, I'm comparing the <code>availablePods</code> with the <code>deployedPods</code> and if they don't match, I generate an alert. </p>

<p><strong>How can I replicate this logic using Prometheus using Alerting Rules and/or Alertmanager config, where it checks the number of available pod instances for a given app or job, and if it doesn't match a specified number of instances, it will trigger an alert?</strong></p>

<p>The specified threshold can be total <code>deployedPods</code> or it can come from another config file or template.</p>

<p>I don’t know how to do this for all namespaces but for one namespace it will look like:</p>

<pre><code>curl -k -s 'https://prometheus-k8s/api/v1/query?query=(sum(kube_deployment_spec_replicas%7Bnamespace%3D%22default%22%7D)%20without%20(deployment%2C%20instance%2C%20pod))%20-%20(sum(kube_deployment_status_replicas_available%7Bnamespace%3D%22default%22%7D)%20without%20(deployment%2C%20instance%2C%20pod))'
</code></pre>

<p>This is curl request to the default namespace.</p>

<p>Alert config will look like:</p>

<pre><code>groups:
- name: example
  rules:

  # Alert for any instance that is unreachable for &gt;5 minutes.
  - alert: availablePods!=deployedPods
    expr: (sum(kube_deployment_spec_replicas{namespace="$Name_of_namespace"}) without (deployment, instance, pod)) - (sum(kube_deployment_status_replicas_available{namespace="$Name_of_namespace"}) without (deployment, instance, pod)) != 0
    for: 15m
    labels:
      severity: page
    annotations:
      summary: "availablePods are not equal deployedPods"
      description: "In namespace $Name_of_namespace more than 15 minutes availablePods are not equal deployedPods. "
</code></pre>

<p>Don’t forget to change variable <code>$Name_of_namespace</code> to namespace name where you want to check.</p>
