<p>I am trying to use both kubernetes HPA and cluster-autoscaler in production, but I want watch when the scaling is triggerred so that I can notify ops and developers.
Are there any webhook I can custom in the scaling lifecycle, or which event should I watch since
 web already collected the event to a isolate ES?</p>

Peng Liu
<p>I would use <a href="https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/" rel="nofollow noreferrer">Container Lifecycle Hooks</a>, to be more specific the <code>PostStart</code> and <code>PreStop</code>.</p>

<p>In Kubernetes documentation we can read the following:</p>

<blockquote>
  <p>There are two hooks that are exposed to Containers:</p>
  
  <p><code>PostStart</code></p>
  
  <p>This hook executes immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT. No parameters are passed to the handler.</p>
  
  <p><code>PreStop</code></p>
  
  <p>This hook is called immediately before a container is terminated due to an API request or management event such as liveness probe failure, preemption, resource contention and others. A call to the preStop hook fails if the container is already in terminated or completed state. It is blocking, meaning it is synchronous, so it must complete before the call to delete the container can be sent. No parameters are passed to the handler.</p>
  
  <p>A more detailed description of the termination behavior can be found in  <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods" rel="nofollow noreferrer">Termination of Pods</a>.</p>
</blockquote>

<p>You could use those to execute a specific command, like <code>run.sh</code>, or to execute an HTTP request against a specific end point on the Container.</p>

<p>Example <code>pod</code> might look like the following:</p>

<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: lifecycle-demo
spec:
  containers:
  - name: lifecycle-demo-container
    image: nginx
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler &gt; /usr/share/message"]
      preStop:
        exec:
          command: ["/bin/sh","-c","nginx -s quit; while killall -0 nginx; do sleep 1; done"]
</code></pre>

<p>As for Cluster scaling you might monitor the events from Apiserver or from command line.
If you do <code>kubectl get event</code> and nodes will scale you might see something like:</p>

<pre><code>$ kubectl get event --watch
LAST SEEN   TYPE     REASON                    KIND   MESSAGE
5m26s       Normal   RegisteredNode            Node   Node gke-standard-cluster-1-default-pool-a663f7f4-8dxk event: Registered Node gke-standard-cluster-1-default-pool-a663f7f4-8dxk in Controller
5m26s       Normal   RegisteredNode            Node   Node gke-standard-cluster-1-default-pool-a663f7f4-pk6v event: Registered Node gke-standard-cluster-1-default-pool-a663f7f4-pk6v in Controller
5m26s       Normal   RegisteredNode            Node   Node gke-standard-cluster-1-default-pool-a663f7f4-wt5m event: Registered Node gke-standard-cluster-1-default-pool-a663f7f4-wt5m in Controller
13m         Normal   Starting                  Node   Starting kubelet.
13m         Normal   NodeHasSufficientMemory   Node   Node gke-standard-cluster-1-pool-1-557d58bb-73x0 status is now: NodeHasSufficientMemory
13m         Normal   NodeHasNoDiskPressure     Node   Node gke-standard-cluster-1-pool-1-557d58bb-73x0 status is now: NodeHasNoDiskPressure
13m         Normal   NodeHasSufficientPID      Node   Node gke-standard-cluster-1-pool-1-557d58bb-73x0 status is now: NodeHasSufficientPID
13m         Normal   NodeAllocatableEnforced   Node   Updated Node Allocatable limit across pods
13m         Normal   NodeReady                 Node   Node gke-standard-cluster-1-pool-1-557d58bb-73x0 status is now: NodeReady
13m         Normal   Starting                  Node   Starting kube-proxy.
13m         Normal   RegisteredNode            Node   Node gke-standard-cluster-1-pool-1-557d58bb-73x0 event: Registered Node gke-standard-cluster-1-pool-1-557d58bb-73x0 in Controller
5m26s       Normal   RegisteredNode            Node   Node gke-standard-cluster-1-pool-1-557d58bb-73x0 event: Registered Node gke-standard-cluster-1-pool-1-557d58bb-73x0 in Controller
13m         Normal   Starting                  Node   Starting kubelet.
13m         Normal   NodeHasSufficientMemory   Node   Node gke-standard-cluster-1-pool-1-557d58bb-fbpz status is now: NodeHasSufficientMemory
13m         Normal   NodeHasNoDiskPressure     Node   Node gke-standard-cluster-1-pool-1-557d58bb-fbpz status is now: NodeHasNoDiskPressure
13m         Normal   NodeHasSufficientPID      Node   Node gke-standard-cluster-1-pool-1-557d58bb-fbpz status is now: NodeHasSufficientPID
13m         Normal   NodeAllocatableEnforced   Node   Updated Node Allocatable limit across pods
13m         Normal   NodeReady                 Node   Node gke-standard-cluster-1-pool-1-557d58bb-fbpz status is now: NodeReady
13m         Normal   RegisteredNode            Node   Node gke-standard-cluster-1-pool-1-557d58bb-fbpz event: Registered Node gke-standard-cluster-1-pool-1-557d58bb-fbpz in Controller
13m         Normal   Starting                  Node   Starting kube-proxy.
5m26s       Normal   RegisteredNode            Node   Node gke-standard-cluster-1-pool-1-557d58bb-fbpz event: Registered Node gke-standard-cluster-1-pool-1-557d58bb-fbpz in Controller
13m         Normal   Starting                  Node   Starting kubelet.
13m         Normal   NodeHasSufficientMemory   Node   Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 status is now: NodeHasSufficientMemory
13m         Normal   NodeHasNoDiskPressure     Node   Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 status is now: NodeHasNoDiskPressure
13m         Normal   NodeHasSufficientPID      Node   Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 status is now: NodeHasSufficientPID
13m         Normal   NodeAllocatableEnforced   Node   Updated Node Allocatable limit across pods
13m         Normal   NodeReady                 Node   Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 status is now: NodeReady
13m         Normal   Starting                  Node   Starting kube-proxy.
13m         Normal   RegisteredNode            Node   Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 event: Registered Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 in Controller
5m26s       Normal   RegisteredNode            Node   Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 event: Registered Node gke-standard-cluster-1-pool-1-557d58bb-v5v5 in Controller
</code></pre>
