title: Advice for how to manage python modules in docker?
body: <p>I am after advice on how to manage python modules within the context of docker. </p>
  
  <p>Current options that I'm aware of include:  </p>
  
  <ol>
  <li>Installing them individually via pip in the build process  </li>
  <li>Installing them together via pip in the build process via <code>requirments.txt</code>  </li>
  <li>Installing them to a volume and adding the volume to the PYTHONPATH</li>
  </ol>
  
  <p>Ideally I want a solution that is fully re-producible and that doesn't require every module to be re-installed if I decide to add another module or update the version of one of them.  </p>
  
  <p>From my perspective:<br>
  (2) is an issue because the docker <code>ADD</code> command  (to get access to the <code>requirements.txt</code> file) apparently invalidates the cache and means that any changes to the file means everything has to be re-built / re-installed everytime you build the image.<br>
  (1) keeps the cache intact but means you'd need to specify the exact version for each package (and potentially their dependencies?) which seems like it could be pretty tedious and error prone.<br>
  (3) is currently my personal favorite as it allows the packages to persist between images/builds and allows for <code>requirements.txt</code> to be used. Only downside is that essentially you are storing the packages on your local machine rather than the image which leads to the container being dependent on the host OS which kind-of defeats the point of a container.</p>
  
  <p>So yer I'm not entirely sure what best practices are here and would appreciate advice. </p>
  
  <p>For reference there have been other questions on this topic but I don't feel any of them properly address my above question:<br>
  <a href="https://stackoverflow.com/questions/52896954/docker-with-modified-python-modules">docker with modified python modules?</a><br>
  <a href="https://stackoverflow.com/questions/41381350/docker-compose-installing-requirements-txt">Docker compose installing requirements.txt</a><br>
  <a href="https://stackoverflow.com/questions/44339375/how-can-i-install-python-modules-in-a-docker-image">How can I install python modules in a docker image?</a>  </p>
  
  <p>EDIT:<br>
  Just some additional notes to give some more context. My projects are typically data analysis focused (rather than software development or web development). I tend to use multiple images (1 for python, 1 for R, 1 for the database) using docker compose to manage them all together.  So far I've been using a makefile on the host OS to re-build the project from scratch i.e. something like</p>
  
  <pre><code>some_output.pdf:  some_input.py
      docker-compose run python_container python some_input.py
  </code></pre>
  
  <p>where the outputs are written to a volume on the host OS</p>