title: Dataproc + python package: Distribute updated versions
body: <p>Currently I am developing a Spark application on Google DataProc. Frequently, I need to update the Python package. During provisioning I run the following commands:</p>
  
  <pre><code>echo "Downloading and extracting source code..."
  gsutil cp gs://mybucket/mypackage.tar.gz ./
  tar -xvzf mypackage.tar.gz
  cd ./mypackage
  
  echo "Installing requirements..."
  sudo apt-get install -y python-pip
  python setup.py install
  </code></pre>
  
  <p>However, what is the most effective way to distribute updated packages within the cluster? Is there any automation already built-in (like Chef e.g. does it)?</p>
  
  <p>Currently, I do two different things: Deploy and bootstrap a new cluster (takes time) or SSH to each node and copy + install the updated package.</p>