{"action":"created","issue":{"url":"https://api.github.com/repos/dask/daskernetes/issues/45","repository_url":"https://api.github.com/repos/dask/daskernetes","labels_url":"https://api.github.com/repos/dask/daskernetes/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/dask/daskernetes/issues/45/comments","events_url":"https://api.github.com/repos/dask/daskernetes/issues/45/events","html_url":"https://github.com/dask/daskernetes/issues/45","id":296956192,"number":45,"title":"Exporting an environment","user":{"login":"limx0","id":4816153,"avatar_url":"https://avatars2.githubusercontent.com/u/4816153?v=4","gravatar_id":"","url":"https://api.github.com/users/limx0","html_url":"https://github.com/limx0","followers_url":"https://api.github.com/users/limx0/followers","following_url":"https://api.github.com/users/limx0/following{/other_user}","gists_url":"https://api.github.com/users/limx0/gists{/gist_id}","starred_url":"https://api.github.com/users/limx0/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/limx0/subscriptions","organizations_url":"https://api.github.com/users/limx0/orgs","repos_url":"https://api.github.com/users/limx0/repos","events_url":"https://api.github.com/users/limx0/events{/privacy}","received_events_url":"https://api.github.com/users/limx0/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-14T02:30:36Z","updated_at":"2018-02-14T02:48:52Z","closed_at":"2018-02-14T02:48:52Z","author_association":"NONE","body":"Is it worth adding a couple of util scripts that would export the users current environment (conda or virtual env) and builds a docker image on top of `daskdev/dask:latest`? \r\n\r\nBeing able to pass a list of conda/pip packages is fine for relatively simple environments / prototyping but I can see value in something slightly more stable. Building a new image (Shouldn't need to be done too often) will increase the connection time to the KubeCluster, but will reduce the worker start up time.\r\n\r\nI have some basic POC of this, which I am currently using, which looks something like\r\n- On Jupyterlab, build my conda env / validate locally in notebook\r\n-  Export the environment\r\n- Build a docker image on top of `daskdev/dask:latest`, using something like\r\n\r\n```python\r\ndockerfile_template = (\r\n    'FROM daskdev/dask:latest\\n'\r\n    'ADD {environment_file} /opt/app/environment.yml\\n'\r\n    'RUN /opt/conda/bin/conda env update -n dask -f /opt/app/environment.yml && \\ \\n'\r\n    '    conda clean -tipsy'\r\n)\r\n\r\ndef build_publish_dockerfile(context_dir, dockerfile_txt, tag):\r\n    with pathlib.Path(os.getcwd()).joinpath('dockerfile').open('w') as f:\r\n        f.write(dockerfile_txt)\r\n    client.images.build(\r\n        path='.', dockerfile='dockerfile', tag='%s/%s' % (DOCKER_HUB_REPO, tag), nocache=True\r\n    )\r\n\r\n\r\ndef image_from_conda_env(env_name, tag, conda_bin='conda'):\r\n    with tempfile.TemporaryDirectory() as tmp_dir:\r\n        env_file = pathlib.Path(tmp_dir).joinpath('environment.yml')\r\n        export_conda_env(env_name, env_file, conda_bin)\r\n        dockerfile = dockerfile_template.format(env_file)\r\n        build_publish_dockerfile(tmp_dir, dockerfile_txt=dockerfile, tag=tag)\r\n\r\nimage_from_conda_env('myenv', 'dask-worker-myenv')\r\n\r\nk = KubeCluster(image='dask-worker-myenv')\r\n```\r\n\r\nIs this in the works? Or any thoughts on the above?\r\n"},"comment":{"url":"https://api.github.com/repos/dask/daskernetes/issues/comments/365478987","html_url":"https://github.com/dask/daskernetes/issues/45#issuecomment-365478987","issue_url":"https://api.github.com/repos/dask/daskernetes/issues/45","id":365478987,"user":{"login":"limx0","id":4816153,"avatar_url":"https://avatars2.githubusercontent.com/u/4816153?v=4","gravatar_id":"","url":"https://api.github.com/users/limx0","html_url":"https://github.com/limx0","followers_url":"https://api.github.com/users/limx0/followers","following_url":"https://api.github.com/users/limx0/following{/other_user}","gists_url":"https://api.github.com/users/limx0/gists{/gist_id}","starred_url":"https://api.github.com/users/limx0/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/limx0/subscriptions","organizations_url":"https://api.github.com/users/limx0/orgs","repos_url":"https://api.github.com/users/limx0/repos","events_url":"https://api.github.com/users/limx0/events{/privacy}","received_events_url":"https://api.github.com/users/limx0/received_events","type":"User","site_admin":false},"created_at":"2018-02-14T02:48:52Z","updated_at":"2018-02-14T02:48:52Z","author_association":"NONE","body":"Thats very useful, thanks @mrocklin. Closing for now. "}}
{"actor":{"display_login":"limx0"}}