<p>After upgrading Jenkins to version <code>2.375.4</code> and Kubernetes AWS EKS cluster to <code>v1.23</code> along with changing container runtime from <code>docker</code> to <code>containerd</code>, I sometimes get the following error on Jenkins jobs that run on Kubernetes AWS EKS cluster via Jenkins agent (Jenkins slave).</p>
<p>Below is the error I get:</p>
<pre><code>03:39:51  java.nio.channels.ClosedChannelException
03:39:51  Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip-10-20-53-103.eu-west-1.compute.internal/10.20.53.103:38004
03:39:51        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
03:39:51        at hudson.remoting.Request.call(Request.java:199)
03:39:51        at hudson.remoting.Channel.call(Channel.java:999)
03:39:51        at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:153)
03:39:51        at jdk.internal.reflect.GeneratedMethodAccessor1121.invoke(Unknown Source)
03:39:51        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
03:39:51        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
03:39:51        at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:138)
03:39:51        at com.sun.proxy.$Proxy262.execute(Unknown Source)
03:39:51        at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1359)
03:39:51        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
03:39:51        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
03:39:51        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
03:39:51        at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
03:39:51        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
03:39:51        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
03:39:51  Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: df677487-c98d-4870-aa71-74faab41e552
03:39:51  Also:   org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for current-frontend-e2e-native-deps-9106-4ptkj-3hltr-q0stv; current-frontend-e2e-native-deps-9106-4ptkj-3hltr-q0stv was marked offline: Connection was broken
03:39:51        at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:182)
03:39:51        at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:154)
03:39:51        at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$Translator.get(ExecutorStepDynamicContext.java:147)
03:39:51        at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:164)
03:39:51        at org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:154)
03:39:51        at org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:95)
03:39:51        at org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:139)
03:39:51        at org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:137)
03:39:51        at org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297)
03:39:51        at org.jenkinsci.plugins.workflow.cps.CpsBodySubContext.doGet(CpsBodySubContext.java:88)
03:39:51        at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:75)
03:39:51        at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Callback.finished(CoreWrapperStep.java:187)
03:39:51        at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Execution2$Callback2.finished(CoreWrapperStep.java:150)
03:39:51        at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution$TailCall.lambda$onFailure$1(GeneralNonBlockingStepExecution.java:156)
03:39:51        at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution.lambda$run$0(GeneralNonBlockingStepExecution.java:77)
03:39:51        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
03:39:51        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
03:39:51  Caused: hudson.remoting.RequestAbortedException
03:39:51    at hudson.remoting.Request.abort(Request.java:346)
03:39:51    at hudson.remoting.Channel.terminate(Channel.java:1080)
03:39:51    at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:241)
03:39:51    at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:221)
03:39:51    at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
03:39:51    at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:289)
03:39:51    at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:168)
03:39:51    at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:825)
03:39:51    at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:155)
03:39:51    at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:143)
03:39:51    at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)
03:39:51    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:30)
03:39:51    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:70)
03:39:51    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
03:39:51    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
03:39:51    at java.base/java.lang.Thread.run(Thread.java:829)
</code></pre>
<p>What is the reason for it? How to fix it?</p>

Abdullah Khawer
<p><strong>Possible Solutions:</strong></p>
<ol>
<li><p>Make sure that your <code>kubernetes-plugin</code> is on the latest version and not outdated.</p>
</li>
<li><p>Make sure that the <code>java</code> version of your Jenkins master matched with the <code>java</code> version of Jenkins slave to avoid any incompatibilities.</p>
</li>
<li><p>Make sure that the pod is not throttling and has enough CPU and/or Memory. If not, increase one of them or both of them to fix this issue.</p>
</li>
</ol>
<p><strong>How did I find solution no. 3?</strong></p>
<p>Looking at the metrics of the containers of that job pod on Grafana, I realised that CPU usage reached 100% that caused CPU throttling for the jnlp container. Increasing its CPU request and limit fixed the issue.</p>
<p>Old Configuration:</p>
<pre class="lang-yaml prettyprint-override"><code>resources:
  limits:
    cpu: &quot;2&quot;
    memory: &quot;2Gi&quot;
  requests:
    cpu: &quot;2&quot;
    memory: &quot;2Gi&quot;
</code></pre>
<p>New Configuration:</p>
<pre class="lang-yaml prettyprint-override"><code>resources:
  limits:
    cpu: &quot;3&quot;
    memory: &quot;2Gi&quot;
  requests:
    cpu: &quot;3&quot;
    memory: &quot;2Gi&quot;
</code></pre>
