<p>Using <a href="https://helm.dask.org/" rel="nofollow noreferrer">Helm</a>, I've created a Dask cluster.</p>

<pre><code>NAME                               READY   STATUS    RESTARTS   AGE
dask01-jupyter-aaa-aaaa            1/1     Running   0          3d19h
dask01-scheduler-bbb-bbbb          1/1     Running   0          3d19h
dask01-worker-ccc-cccc             1/1     Running   0          3d19h
dask01-worker-ddd-dddd             1/1     Running   0          3d19h
dask01-worker-eee-eeee             1/1     Running   0          3d19h
</code></pre>

<p>I can run a basic Dask workload.</p>

<pre class="lang-py prettyprint-override"><code>import dask.array as da

array = da.ones((1000, 1000, 1000), chunks=(100, 100, 10))
</code></pre>

<p>Now, I would like to connect it to a client somehow:</p>

<pre class="lang-py prettyprint-override"><code>from dask import distributed

cluster = None # TODO: configure KubeCluster somehow https://kubernetes.dask.org/en/latest/
client = distributed.Client(cluster)
</code></pre>

<p>This works if I want to <a href="https://kubernetes.dask.org/en/latest/#quickstart" rel="nofollow noreferrer">launch a cluster</a>:</p>

<pre class="lang-py prettyprint-override"><code>from dask_kubernetes import KubeCluster

cluster = KubeCluster.from_yaml('worker-spec.yml')
</code></pre>

<p>But how do I connect to an existing cluster?</p>

Seanny123
<p>If you've already installed the Dask Helm package, then you can use <code>kubectl</code> to retrieve the TCP connection address to pass to <code>distributed.Client</code>. If you chose to, for example, use a Helm release name of <code>dask-abc</code>distributed. in <code>helm install</code> - see <a href="https://helm.sh/docs/intro/using_helm/#helm-install-installing-a-package" rel="nofollow noreferrer">here</a> - when setting up the cluster, then you could follow the <a href="https://docs.dask.org/en/latest/setup/kubernetes-helm.html#connect-to-dask-and-jupyter" rel="nofollow noreferrer">Dask Helm and Kubernetes docs</a> and use <code>kubectl get services</code> with <a href="https://kubernetes.io/docs/reference/kubectl/jsonpath/" rel="nofollow noreferrer"><code>jsonpath</code></a> to filter the output of this command and retrieve only the IP address of the Dask scheduler service (which will be named <code>dask-abc-scheduler</code>).</p>

<p><a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicationcontroller/#running-an-example-replicationcontroller" rel="nofollow noreferrer">Here</a> is a similar example of using <code>kubectl get pods</code> with <code>jsonpath</code> to get the names of pods (see the line starting with <code>pods=$(</code>). You'll have to wait until the IP address of the service becomes available, using the <code>--watch</code> flag (see the <strong>Note</strong> section from <a href="https://kubernetes.io/docs/reference/kubectl/overview/#examples-common-operations" rel="nofollow noreferrer">here</a>)</p>

<pre class="lang-sh prettyprint-override"><code>$ export RELEASE_NAME=dask-abc

# wait until load balancer EXTERNAL_IP is available
$ kubectl get services --wait $RELEASE_NAME-scheduler

# get Dask scheduler address
$ dask_scheduler=$(kubectl get services \
      $RELEASE_NAME-scheduler \
      --output=jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ echo $dask_scheduler
</code></pre>

<p>Then, the <code>dask_scheduler</code> address printed above can be used in the url that is passed to <code>distributed.Client()</code> in your Python code</p>

<pre class="lang-py prettyprint-override"><code>&gt; client_connection_url = "tcp://&lt;dask-scheduler&gt;:8786"
&gt; client = distributed.client(client_connection_url)
&gt; print(client)
.
.
.
.
</code></pre>
