<p>Platform: AWS EKS</p>

<p>Output of helm version:</p>

<pre><code>Client: &amp;version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Server: &amp;version.Version{SemVer:"v2.14.2", GitCommit:"a8b13cc5ab6a7dbef0a58f5061bcc7c0c61598e7", GitTreeState:"clean"}
</code></pre>

<p>Output of kubectl version:</p>

<pre><code>Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T09:18:22Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.10-eks-2e569f", GitCommit:"2e569fd887357952e506846ed47fc30cc385409a", GitTreeState:"clean", BuildDate:"2019-07-25T23:13:33Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.): AWS EKS
</code></pre>

<p><strong>The problem:</strong> 
After jenkins pod restart, the pod got a new IP address and ReadinesProbe supposed to update the endpoints but it doesn't.</p>

<pre><code>kubectl get endpoints
jenkins                                  &lt;none&gt;
jenkins-agent                            &lt;none&gt;
</code></pre>

<p><strong>Error:</strong> </p>

<pre><code>Readiness probe failed: Get http://192.168.0.109:8080/login: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
</code></pre>

<p>I can successfully access above URL from all pods and worker nodes and I get correct Headers.</p>

<p>This happened after helm failed to upgrade jenkins and then I rollback the release, and it was successful (apart from now not updating endpoints) 
Now I need to edit endpoints manually to point Endpoints to the correct IP address of a pod.</p>

<p>Current ReadinesProbe from deployment is:</p>

<pre><code>    readinessProbe:
      failureThreshold: 3
      httpGet:
        path: /login
        port: http
        scheme: HTTP
      initialDelaySeconds: 60
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 1
</code></pre>

<p><strong>Log from Jenkins pod is:</strong></p>

<pre><code>Events:
  Type     Reason                  Age    From                                                  Message
  ----     ------                  ----   ----                                                  -------
  Normal   Scheduled               8m13s  default-scheduler                                     Successfully assigned default/jenkins-pod-id to &lt;ip&gt;.&lt;region&gt;.compute.internal
  Normal   SuccessfulAttachVolume  8m6s   attachdetach-controller                               AttachVolume.Attach succeeded for volume "jenkins"
  Normal   Pulling                 8m4s   kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  pulling image "jenkins/jenkins:2.176.2-alpine"
  Normal   Pulled                  7m57s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Successfully pulled image "jenkins/jenkins:2.176.2-alpine"
  Normal   Created                 7m56s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Created container
  Normal   Started                 7m56s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Started container
  Normal   Pulling                 7m43s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  pulling image "jenkins/jenkins:2.176.2-alpine"
  Normal   Pulled                  7m42s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Successfully pulled image "jenkins/jenkins:2.176.2-alpine"
  Normal   Created                 7m42s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Created container
  Normal   Started                 7m42s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Started container
  Warning  Unhealthy               6m40s  kubelet, &lt;ip&gt;.&lt;region&gt;.compute.internal  Readiness probe failed: Get http://&lt;IP&gt;:8080/login: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
</code></pre>

<p>The pod got IP almost instantly but it takes a few minutes for the container to start. How can I get the ReadinesProbe to update Endpoints or even get ReadinesProbe logs? This is running in AWS so no access to the controller to get more logs.</p>

<p>If I update the endpoints fast enough, the ReadinesProbe won't fail but this doesn't help when the pod restarts next time.</p>

<p><strong>Update:</strong>
Just enabled EKS logs and got this:</p>

<pre><code>deployment_controller.go:484] Error syncing deployment default/jenkins: Operation cannot be fulfilled on deployments.apps "jenkins": the object has been modified; please apply your changes to the latest version and try again
</code></pre>

tr53
<p>Below helped. The Readiness probe is still failing but this is due to Jenkins taking 90s to start. I will update this.</p>

<pre><code>helm delete jenkins
release "jenkins" deleted


helm rollback jenkins 25
Rollback was a success! Happy Helming!
</code></pre>
