<p>I am researching the K8s architecture, focusing on the flow of pod spinning in the system.</p>
<p>I was wondering <em><strong>how</strong></em> (that is, who is initiating the communication) and <em><strong>when</strong></em> the different components of the control plane communicate with each other.</p>
<p>I have followed the excellent talk of <strong>Jérôme Petazzoni</strong> at LISA 2019 (<a href="https://www.youtube.com/watch?v=3KtEAa7_duA" rel="nofollow noreferrer">here</a>) to understand the architecture of the control plane, and read the concepts on <a href="https://kubernetes.io/docs/concepts/architecture" rel="nofollow noreferrer">kubernetes.io</a>.</p>
<p>However, I still haven't found the answers to the following questions:</p>
<ol>
<li>Who initiates the resource check of each node, in the <a href="https://kubernetes.io/docs/concepts/architecture/nodes/#node-capacity" rel="nofollow noreferrer">documentation</a> it is written:</li>
</ol>
<blockquote>
<p>Node objects track information about the Node's resource capacity: for example, the amount of memory available and the number of CPUs. Nodes that self-register report their capacity during registration. If you manually add a Node, then you need to set the node's capacity information when you add it.</p>
</blockquote>
<p>However, there is no specification on when does it update at <code>etcd</code>, and who initiates the regular update (other than the <em>heartbeat</em> that updates the status of the node).</p>
<p>Also, when does the cache of the scheduler update?</p>
<ol start="2">
<li>Who informs the different components about new pending requests? That is, how is the <code>controller-manager</code>/<code>scheduler</code> <em>&quot;knows&quot;</em> when it suppose to do its job? Each request is written as a manifest in <code>etcd</code> by the <code>kube-api-server</code>, but these components aren't connected to <code>etcd</code> directly.</li>
</ol>
<p>Does that mean the API-Server needs to inform each component about each new request?</p>
<p>I have many possible answers, but not a concrete confirmation of the real process in current K8s architecture.</p>

<p>Answering your questions:</p>
<p><strong>Who initiates the resource check of each node?</strong></p>
<p>The component responsible for that is <em>&quot;Node Status Manager&quot;</em> which is a sub-control loop of the &quot;<em>SyncLoop&quot;</em> which is a <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/" rel="nofollow noreferrer">kubelet agent</a> component.</p>
<p>The more detailed answer is in this article: <a href="https://aws.plainenglish.io/kubernetes-deep-dive-kubelet-e4527ed56f4c" rel="nofollow noreferrer">Kubernetes Deep Dive: Kubelet</a>:</p>
<blockquote>
<p>As you can see, the core of <code>kubelet</code>’s work is a control loop, namely: <strong>SyncLoop</strong>.</p>
</blockquote>
<blockquote>
<p>For example, the <strong>Node Status Manager</strong> is responsible for responding to changes in the status of the <code>Node</code>, and then collecting the status of the <code>Node</code> and reporting it to the <code>APIServer</code> through Heartbeat.</p>
</blockquote>
<p>There is also a good diagram:</p>
<p><img src="https://pbs.twimg.com/media/DncWSekUYAAlOMY?format=jpg&amp;name=large" alt="" /></p>
<p>Answering second part:</p>
<p><strong>Who informs the different components about new pending requests? That is, how is the <code>controller-manager</code>/<code>scheduler</code>  <em>&quot;knows&quot;</em> when it suppose to do its job?</strong></p>
<p>The components responsible for that are <a href="https://kubernetes.io/docs/concepts/architecture/controller/" rel="nofollow noreferrer">Kubernetes' controllers</a> and  <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/" rel="nofollow noreferrer">Scheduler</a> itself. Good examples and explanations are in this article: <a href="https://github.com/jamiehannaford/what-happens-when-k8s" rel="nofollow noreferrer">What happens when ... Kubernetes edition!</a>,</p>
<p>Basically after Kubernetes verified the request (authentication, authorization, admission control stuff), it is saved to datastore (<code>etcd</code>), and then it's taken by <a href="https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#initializers" rel="nofollow noreferrer">initializers</a> which may perform some additional logic on the resource (not always), after that it's visible via kube-server. Main part that may interest you is <a href="https://github.com/jamiehannaford/what-happens-when-k8s#control-loops" rel="nofollow noreferrer">Control loops</a>. They are constantly checking if a new request exists in a datastore, and if yes they are proceeding. Example - when you are deploying a new deployment:</p>
<ul>
<li><a href="https://github.com/jamiehannaford/what-happens-when-k8s#deployments-controller" rel="nofollow noreferrer">Deployments controller</a> is taking a request - it will realise that there is no ReplicaSet record associated, it will roll-out new one</li>
<li><a href="https://github.com/jamiehannaford/what-happens-when-k8s#replicasets-controller" rel="nofollow noreferrer">ReplicaSets controller</a>, like deployments controller, it will take a request and roll-out new pods</li>
<li>Pods are ready but they are in pending state - now <a href="https://github.com/jamiehannaford/what-happens-when-k8s#scheduler" rel="nofollow noreferrer">Scheduler</a> (which is like previous controllers, listening constantly for new requests from the data store - it's de facto answer for your question) will find a suitable node and schedule a pod to a node. Now, kubelet agent on the node will create a new pod.</li>
</ul>
<p>For more details I'd strongly suggest reading the earlier mentioned article - <a href="https://github.com/jamiehannaford/what-happens-when-k8s" rel="nofollow noreferrer">What happens when ... Kubernetes edition!</a>.</p>
<p><strong>Does that mean the API-Server needs to inform each component about each new request?</strong></p>
<p>It works in a different way - the kube-apiserver is making requests to be visible, and controllers, which are loops, are detecting new requests and starting to proceed with them.</p>
