<p>This is a purely theoretical question. A standard Kubernetes clusted is given with autoscaling in place. If memory goes above a certain targetMemUtilizationPercentage than a new pod is started and it takes on the flow of requests that is coming to the contained service. The number of minReplicas is set to 1 and the number of maxReplicas is set to 5.</p>
<p>What happens when the number of pods that are online reaches maximum (5 in our case) and requests from clients are still coming towards the node? Are these requests buffered somewhere of they are discarded? Can I take any actions to avoid request loss?</p>

otto
<p>Natively Kubernetes does not support messaging queue buffering. Depends on the scenario and setup you use your requests will most likely 'timeout'.  To efficiently manage those you`ll need custom resource running inside Kubernetes cluster.</p>
<p>In that situations it very common to use a message broker which ensures  communication between microservices is reliable and stable, that the messages are managed and monitored within the system and that messages don’t get lost.</p>
<p><a href="https://www.rabbitmq.com/" rel="nofollow noreferrer">RabbitMQ</a>, <a href="https://kafka.apache.org/" rel="nofollow noreferrer">Kafka</a> and <a href="https://redis.io/" rel="nofollow noreferrer">Redis</a> appears to be most popular but choosing the right one will heaving depend on your requirement and features needed.</p>
<p>Worth to note since Kubernetes essentially runs on linux is that linux itself also manages/limits the requests coming in socket. You may want to read more about it <a href="https://man7.org/linux/man-pages/man2/listen.2.html" rel="nofollow noreferrer">here</a>.</p>
<p>Another thing is that if you have pods limits set or lack of resource it is most likely that pods might be restarted or cluster will become unstable. Usually you can prevent it by configuring some kind of &quot;circuit breaker&quot; to limit amount of requests that could go to backed without overloading it. If the amount of requests goes beyond the circuit breaker threshold, excessive requests will be dropped.</p>
<p>It is better to drop some request than having <a href="https://en.wikipedia.org/wiki/Cascading_failure" rel="nofollow noreferrer">cascading failure</a>.</p>
