title: How to omit Promql label in prometheus to avoid duplicate notifications
body: <p>I have a multizone kubernetes cluster having 3 zones with prometheus monitoring deployed with replicaset=3. Each zone has 1 alertmanager,1 kubestate metrics,1 grafana &amp; 1 prometheus operator pod running.</p>
  
  <p>When I am getting alerts for example, an alert DeploymentReplicasNotUpdated , I am getting three alerts- one from each kube-state-metric pod.Alerts are differing only by instance label,which is ip address of kube-state-metric pod.
  alert format given below </p>
  
  <pre><code>alertname="DeploymentReplicasNotUpdated" app="kube-state-metrics" deployment="mydeployment" instance="172.30.83.127:8080" job="kubernetes-service-endpoints" kubernetes_name="kube-state-metrics" kubernetes_namespace="mykubenamespace" namespace="myns" severity="critical"
  
  alertname="DeploymentReplicasNotUpdated" app="kube-state-metrics" deployment="mydeployment" instance="172.30.83.125:8080" job="kubernetes-service-endpoints" kubernetes_name="kube-state-metrics" kubernetes_namespace="mykubenamespace" namespace="myns" severity="critical"
  
  alertname="DeploymentReplicasNotUpdated" app="kube-state-metrics" deployment="mydeployment" instance="172.30.83.127:8080" job="kubernetes-service-endpoints" kubernetes_name="kube-state-metrics" kubernetes_namespace="mykubenamespace" namespace="myns" severity="critical"
  </code></pre>
  
  <p>Could someone please help me in grouping these 3 alerts into single alert in alertmanager since they refer to same alert ? Basically I need to exclude instance label  as shown and return only 1 alert instead of 3</p>
  
  <pre><code>alertname="DeploymentReplicasNotUpdated" app="kube-state-metrics" deployment="mydeployment"job="kubernetes-service-endpoints" kubernetes_name="kube-state-metrics" kubernetes_namespace="mykubenamespace" namespace="myns" severity="critical"
  </code></pre>
  
  <p>promql given below   </p>
  
  <pre><code> ((kube_deployment_status_replicas_updated != kube_deployment_spec_replicas)
        or (kube_deployment_status_replicas_available != kube_deployment_spec_replicas))
        unless (kube_deployment_spec_paused == 1)
  </code></pre>