<p>I'm using Dask deployed using Helm on a Kubernetes cluster in Kubernetes Engine on GCP. My current cluster set up has 5 nodes with each node having 8 cpus, 30 gb:</p>

<p>I ran the notebook named <code>05-nyc-taxi.ipynb</code>, which resulted in workers getting killed.</p>

<p>When I restarted the Dask client it shows that I now have zero workers and zero memory:</p>

<p><a href="https://i.stack.imgur.com/HmfyA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/HmfyA.png" alt="enter image description here"></a></p>

<p>However, when I run <code>kubectl get services</code> and <code>kubectl get pods</code>, it shows that my pods and services are running:</p>

<p><a href="https://i.stack.imgur.com/dJKww.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dJKww.png" alt="enter image description here"></a></p>

<p>Any idea why this might be the case?</p>

<p>When you restart the client, is kills all the workers, and starts making new ones. That process is asynchronous, but the rendering of the client object happens immediately - so there are no workers at that moment. You could render the client object again (and again) later:</p>

<pre><code>In[]:  client
</code></pre>

<p>Or check the dashboard.</p>

<p>Or, better, you could render the cluster object itself which, so long as you have jupyter widgets installed in the environment, will update itself in real-time. If you didn't happen to assign your cluster object before, it will also be available as <code>client.cluster</code>.</p>

<p>btw: <em>why</em> are you having to restart the cluster like this?</p>
