{"action":"created","issue":{"url":"https://api.github.com/repos/singularityhub/sregistry/issues/90","repository_url":"https://api.github.com/repos/singularityhub/sregistry","labels_url":"https://api.github.com/repos/singularityhub/sregistry/issues/90/labels{/name}","comments_url":"https://api.github.com/repos/singularityhub/sregistry/issues/90/comments","events_url":"https://api.github.com/repos/singularityhub/sregistry/issues/90/events","html_url":"https://github.com/singularityhub/sregistry/pull/90","id":293923409,"number":90,"title":"Address #82 large container pulls with download streaming","user":{"login":"dctrud","id":4522799,"avatar_url":"https://avatars1.githubusercontent.com/u/4522799?v=4","gravatar_id":"","url":"https://api.github.com/users/dctrud","html_url":"https://github.com/dctrud","followers_url":"https://api.github.com/users/dctrud/followers","following_url":"https://api.github.com/users/dctrud/following{/other_user}","gists_url":"https://api.github.com/users/dctrud/gists{/gist_id}","starred_url":"https://api.github.com/users/dctrud/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dctrud/subscriptions","organizations_url":"https://api.github.com/users/dctrud/orgs","repos_url":"https://api.github.com/users/dctrud/repos","events_url":"https://api.github.com/users/dctrud/events{/privacy}","received_events_url":"https://api.github.com/users/dctrud/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-02-02T15:22:51Z","updated_at":"2018-02-02T17:07:50Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/singularityhub/sregistry/pulls/90","html_url":"https://github.com/singularityhub/sregistry/pull/90","diff_url":"https://github.com/singularityhub/sregistry/pull/90.diff","patch_url":"https://github.com/singularityhub/sregistry/pull/90.patch"},"body":"This PR is to address #82 - there are 3 changes:\r\n\r\n- 682bba3 sets up the Django logger to direct WARNING and above to console by default, so Django errors can then be seen with `docker-compose logs uwsgi`. This is critical to effectively debug download issues that are occuring in Django code. The logs coming out of UWSGI alone are not enough.\r\n\r\n- 3690bac uses the Django `FileResponse` class to serve a container download without requiring it to be read entirely into RAM. Note that underneath this is going to use `wsgi.filewrapper` as we are running via UWSGI. `FileResponse` is a convenience (vs using the filewrapper and StreamingHttpResponse directly) that was introduced in newish Django versions. We can't use a `with ... as f` context yet, as context manager support for `File.open` was added in Django 2.0 and we are using 1.11\r\n\r\n- bf1e505 disables harakiri for UWSGI. Since we are serving large files via the app/UWSGI we don't know how long is reasonable for a request - there may be a slow connection, a multi GB download could take hours. Auto-kill of unusually long running UWSGI reqests would be appropriate if we further offload serving downloads using xsendfile so that the download itself is handled by nginx, and not our UWSGI  app."},"comment":{"url":"https://api.github.com/repos/singularityhub/sregistry/issues/comments/362645006","html_url":"https://github.com/singularityhub/sregistry/pull/90#issuecomment-362645006","issue_url":"https://api.github.com/repos/singularityhub/sregistry/issues/90","id":362645006,"user":{"login":"vsoch","id":814322,"avatar_url":"https://avatars0.githubusercontent.com/u/814322?v=4","gravatar_id":"","url":"https://api.github.com/users/vsoch","html_url":"https://github.com/vsoch","followers_url":"https://api.github.com/users/vsoch/followers","following_url":"https://api.github.com/users/vsoch/following{/other_user}","gists_url":"https://api.github.com/users/vsoch/gists{/gist_id}","starred_url":"https://api.github.com/users/vsoch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vsoch/subscriptions","organizations_url":"https://api.github.com/users/vsoch/orgs","repos_url":"https://api.github.com/users/vsoch/repos","events_url":"https://api.github.com/users/vsoch/events{/privacy}","received_events_url":"https://api.github.com/users/vsoch/received_events","type":"User","site_admin":false},"created_at":"2018-02-02T17:07:50Z","updated_at":"2018-02-02T17:07:50Z","author_association":"OWNER","body":"Ok here is a log of testing!\r\n\r\n1. I cloned the repository, built and did `docker-compose up -d`\r\n2. An import of `os` needed to be added to logging\r\n3. `docker-compose restart`\r\n4. Log in with twitter\r\n5. Then I shelled into the main and added myself as an admin and superuser\r\n\r\n```\r\ndocker exec -it 5ce7010a2f99 bash\r\n$ python manage.py add_superuser --username vsoch\r\nDEBUG Username: vsoch\r\nDEBUG vsoch is now a superuser.\r\n$ python manage.py add_admin --username vsoch\r\nDEBUG Username: vsoch\r\nDEBUG vsoch can now manage and build.\r\n```\r\nThen I downloaded my credentials at `http://127.0.0.1/token`, updated them at `$HOME/.sregistry` and used singularity global client to push an image:\r\n\r\n```\r\nexport SREGISTRY_CLIENT=registry\r\nregistry sregistry push --name peanut-butter/jelly:time vsoch-hello-world-master-latest.simg \r\n[client|registry] [database|sqlite:////home/vanessa/.singularity/sregistry.db]\r\n[================================] 62/62 MB - 00:00:00\r\n[Return status 201 Created]\r\n```\r\nSearch...\r\n```\r\nsregistry search\r\n sregistry pull peanut-butter/jelly:time\r\n[client|registry] [database|sqlite:////home/vanessa/.singularity/sregistry.db]\r\n[container][update] peanut-butter/jelly:time\r\nSuccess! /home/vanessa/.singularity/shub/peanut-butter-jelly:time.simg\r\n$ sregistry get peanut-butter/jelly:time\r\n/home/vanessa/.singularity/shub/peanut-butter-jelly:time.simg\r\n$ singularity run $(sregistry get peanut-butter/jelly:time)\r\n```\r\nAh yes, so what is missing is the download bar, and this is because the Content-Size is not there. I just added this, and pushed, and I think the PR updated!! I hope I didn't mess anything up. I'm going to try a larger image now.\r\n"}}
{"actor":{"display_login":"vsoch"}}