<p>I am a scientist who is exploring the use of Dask on Amazon Web Services. I have some experience with Dask, but none with AWS.  I have a few large custom task graphs to execute, and a few colleagues who may want to do the same if I can show them how.  I believe that I should be using <a href="https://dask.pydata.org/en/latest/setup/kubernetes-helm.html" rel="nofollow noreferrer">Kubernetes with Helm</a> because I fall into the <a href="https://dask.pydata.org/en/latest/setup/kubernetes.html" rel="nofollow noreferrer">"Try out Dask for the first time on a cloud-based system like Amazon, Google, or Microsoft Azure"</a> category.</p>

<ol>
<li>I also fall into the "Dynamically create a personal and ephemeral deployment for interactive use" category.  Should I be trying native Dask-Kubernetes instead of Helm?  It seems simpler, but it's hard to judge the trade-offs.</li>
<li>In either case, how do you provide Dask workers a uniform environment that includes your own Python packages (not on any package index)?  <a href="https://dask.pydata.org/en/latest/setup/docker.html" rel="nofollow noreferrer">The solution I've found</a> suggests that packages need to be on a <code>pip</code> or <code>conda</code> index.</li>
</ol>

<p>Thanks for any help!</p>

<h3>Use Helm or Dask-Kubernetes ?</h3>

<p>You can use either.  Generally starting with Helm is simpler.</p>

<h3>How to include custom packages</h3>

<p>You can install custom software using pip or conda. They don't need to be on PyPI or the anaconda default channel.  You can point pip or conda to other channels.  Here is an example installing software using pip from github</p>

<pre><code>pip install git+https://github.com/username/repository@branch
</code></pre>

<p>For small custom files you can also use the <a href="http://dask.pydata.org/en/latest/futures.html#distributed.Client.upload_file" rel="nofollow noreferrer">Client.upload_file</a> method.</p>
