<p>I'm encountering a situation where pods are occasionally getting evicted after running out of memory.  Is there any way to set up some kind of alerting where I can be notified when this happens?</p>

<p>As it is, Kubernetes keeps doing its job and re-creating pods after the old ones are removed, and it's often hours or days before I'm made aware that a problem exists at all.</p>

Ameo
<p>GKE exports Kubernetes Events (<code>kubectl get events</code>) to Stackdriver Logging, to the "GKE Cluster Operations" table:
<a href="https://i.stack.imgur.com/SsvHR.png" rel="noreferrer"><img src="https://i.stack.imgur.com/SsvHR.png" alt=""></a></p>

<p>Next, write a query specifically targeting evictions (the query I pasted below might not be accurate):</p>

<p><a href="https://i.stack.imgur.com/AbhF7.png" rel="noreferrer"><img src="https://i.stack.imgur.com/AbhF7.png" alt="enter image description here"></a></p>

<p>Then click "CREATE METRIC" button.</p>

<p>This will create a Log-based Metric. On the left sidebar, click "Logs-based metrics" and click the "Create alert from metric" option on the context menu of this metric:</p>

<p><a href="https://i.stack.imgur.com/BxsKn.png" rel="noreferrer"><img src="https://i.stack.imgur.com/BxsKn.png" alt="enter image description here"></a></p>

<p>Next, you'll be taken to Stackdriver Alerting portal. You can set up alerts there based on thresholds etc.</p>
