<p>I'm installing jupyterhub on k8s using <a href="https://z2jh.jupyter.org/en/stable/jupyterhub/installation.html#install-jupyterhub" rel="nofollow noreferrer">helm</a>.</p>
<pre><code>helm upgrade --cleanup-on-fail --install jupyterhub jupyterhub-2.0.0/jupyterhub/ --namespace my-NS --create-namespace --version=2.0.0 --values my-values.yaml --timeout 30m --debug
</code></pre>
<p>Its failing with error in creating hook-image-awaiter pods.</p>
<p>Error from helm debug:</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>upgrade.go:142: [debug] preparing upgrade for jupyterhub
upgrade.go:150: [debug] performing update for jupyterhub
upgrade.go:322: [debug] creating upgraded release for jupyterhub
client.go:310: [debug] Starting delete for "hook-image-puller" DaemonSet
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "hook-image-awaiter" ServiceAccount
client.go:128: [debug] creating 1 resource(s)
client.go:310: [debug] Starting delete for "hook-image-awaiter" Job
client.go:128: [debug] creating 1 resource(s)
client.go:540: [debug] Watching for changes to Job hook-image-awaiter with timeout of 30m0s
client.go:568: [debug] Add/Modify event for hook-image-awaiter: ADDED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 1, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 2, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 3, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 4, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 5, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
client.go:607: [debug] hook-image-awaiter: Jobs active: 1, jobs failed: 6, jobs succeeded: 0
client.go:568: [debug] Add/Modify event for hook-image-awaiter: MODIFIED
upgrade.go:434: [debug] warning: Upgrade "jupyterhub" failed: pre-upgrade hooks failed: job failed: BackoffLimitExceeded
Error: UPGRADE FAILED: pre-upgrade hooks failed: job failed: BackoffLimitExceeded
helm.go:84: [debug] pre-upgrade hooks failed: job failed: BackoffLimitExceeded
UPGRADE FAILED</code></pre>
</div>
</div>
</p>
<p>There are some answers in web which didn't help to resolve the issue. I have tried following,</p>
<ul>
<li>Increase helm install time out - didn't work</li>
<li>Failed to pull image: I can see jupyterhub/k8s-image-awaiter:2.0.0 got pulled and I do not see any errors when pods are described</li>
</ul>
<p><a href="https://i.stack.imgur.com/Nnscp.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Nnscp.png" alt="enter image description here" /></a></p>
<p>hook-image-puller:
<a href="https://i.stack.imgur.com/L22Oy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/L22Oy.png" alt="enter image description here" /></a>
hook-image-awaiter:
<a href="https://i.stack.imgur.com/iUOkM.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/iUOkM.png" alt="enter image description here" /></a></p>
<p>Event logs:
<a href="https://i.stack.imgur.com/MjOFc.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/MjOFc.png" alt="enter image description here" /></a></p>
<p>PS: I disabled rbac in values.yaml (create=false) and am installing jupyterhub in a new namespace while there is one is running in another namespace)</p>

<p>Disable the pre-pull hook:</p>
<p><a href="https://z2jh.jupyter.org/en/latest/administrator/optimization.html#pulling-images-before-users-arrive" rel="nofollow noreferrer">https://z2jh.jupyter.org/en/latest/administrator/optimization.html#pulling-images-before-users-arrive</a></p>
<p>prePuller:
hook:
enabled: false</p>
