{"action":"created","comment":{"url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/comments/186217041","pull_request_review_id":117761918,"id":186217041,"diff_hunk":"@@ -23,5 +25,21 @@ def wrapper(*args, **kwargs):\n         return wrapper\n \n \n+class pickle_result:","path":"scvi/utils.py","position":10,"original_position":10,"commit_id":"44069abcbb34ec8acba11821957a70b81297a01c","original_commit_id":"44069abcbb34ec8acba11821957a70b81297a01c","user":{"login":"jeff-regier","id":8953355,"avatar_url":"https://avatars2.githubusercontent.com/u/8953355?v=4","gravatar_id":"","url":"https://api.github.com/users/jeff-regier","html_url":"https://github.com/jeff-regier","followers_url":"https://api.github.com/users/jeff-regier/followers","following_url":"https://api.github.com/users/jeff-regier/following{/other_user}","gists_url":"https://api.github.com/users/jeff-regier/gists{/gist_id}","starred_url":"https://api.github.com/users/jeff-regier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeff-regier/subscriptions","organizations_url":"https://api.github.com/users/jeff-regier/orgs","repos_url":"https://api.github.com/users/jeff-regier/repos","events_url":"https://api.github.com/users/jeff-regier/events{/privacy}","received_events_url":"https://api.github.com/users/jeff-regier/received_events","type":"User","site_admin":false},"body":"I see. It's better to keep debugging code locally in a file but it's not a big deal for now since only a few of us our reading the code. Once we move from scvi-dev to scVI, we should keep all the debugging code out of the repo, to make it easier for people to read the code.","created_at":"2018-05-04T20:49:08Z","updated_at":"2018-05-04T20:49:08Z","html_url":"https://github.com/YosefLab/scVI-dev/pull/93#discussion_r186217041","pull_request_url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93","author_association":"COLLABORATOR","_links":{"self":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/comments/186217041"},"html":{"href":"https://github.com/YosefLab/scVI-dev/pull/93#discussion_r186217041"},"pull_request":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93"}},"in_reply_to_id":186202862},"pull_request":{"url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93","id":186057361,"html_url":"https://github.com/YosefLab/scVI-dev/pull/93","diff_url":"https://github.com/YosefLab/scVI-dev/pull/93.diff","patch_url":"https://github.com/YosefLab/scVI-dev/pull/93.patch","issue_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues/93","number":93,"state":"closed","locked":false,"title":"Brain large benchmark","user":{"login":"Edouard360","id":15527397,"avatar_url":"https://avatars3.githubusercontent.com/u/15527397?v=4","gravatar_id":"","url":"https://api.github.com/users/Edouard360","html_url":"https://github.com/Edouard360","followers_url":"https://api.github.com/users/Edouard360/followers","following_url":"https://api.github.com/users/Edouard360/following{/other_user}","gists_url":"https://api.github.com/users/Edouard360/gists{/gist_id}","starred_url":"https://api.github.com/users/Edouard360/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Edouard360/subscriptions","organizations_url":"https://api.github.com/users/Edouard360/orgs","repos_url":"https://api.github.com/users/Edouard360/repos","events_url":"https://api.github.com/users/Edouard360/events{/privacy}","received_events_url":"https://api.github.com/users/Edouard360/received_events","type":"User","site_admin":false},"body":"Brain Large preprocessing was until now, a bit different from Romain's. (In particular, batch indices weren't taken into account). \r\n\r\nRomain's exact preprocessing is at c13928e.If we care exactly about having same train/test dataset, we can look at this previous commit. \r\n\r\nOtherwise, the same results are also reproducible taking the subsample size: 60000 in Romain's notebook.\r\n\r\n```\r\ngene_dataset = BrainLargeDataset(subsample_size=60000)\r\nrun_benchmarks(gene_dataset, n_epochs=250, use_batches=False, tt_split=0.8)\r\n```\r\n\r\nThis will give train/test 48000/12000 ~= 50000/10000 in Romain's notebook. The same final likelihood score + imputation score are achieved  on test set : **LL = 1148, MAE = 1.07**\r\n\r\nIt takes 1119 sec for the 250 epochs. I think Romain does only 120 epochs, and indeed, after 40 epochs we reach the final log likelihood so we could at least reduce that training time to ~180 sec = 3min. with early stopping.\r\n\r\nAs for the Cortex Dataset, 250 epochs of training give  **LL = 1197, MAE = 2.18**, which is good for MAE, and a little above for LL test (~ 1180 in Romain's preprint), even using **adapt_encoder** new function. But imo with a little tuning (learning rate, batch size), Cortex result's are easily reproducible.\r\n\r\nIn terms of timing / GPU usage, **nvidia-smi => 40/50%** due to the `data_loader.next` procedure.\r\n","created_at":"2018-05-04T17:48:50Z","updated_at":"2018-05-04T20:49:08Z","closed_at":"2018-05-04T18:27:36Z","merged_at":"2018-05-04T18:27:36Z","merge_commit_sha":"2476b3ee6aab5a6a6596bb7485c760b1d0669b6d","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93/commits","review_comments_url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93/comments","review_comment_url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/comments{/number}","comments_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues/93/comments","statuses_url":"https://api.github.com/repos/YosefLab/scVI-dev/statuses/44069abcbb34ec8acba11821957a70b81297a01c","head":{"label":"YosefLab:brain-large-benchmark","ref":"brain-large-benchmark","sha":"44069abcbb34ec8acba11821957a70b81297a01c","user":{"login":"YosefLab","id":13837262,"avatar_url":"https://avatars2.githubusercontent.com/u/13837262?v=4","gravatar_id":"","url":"https://api.github.com/users/YosefLab","html_url":"https://github.com/YosefLab","followers_url":"https://api.github.com/users/YosefLab/followers","following_url":"https://api.github.com/users/YosefLab/following{/other_user}","gists_url":"https://api.github.com/users/YosefLab/gists{/gist_id}","starred_url":"https://api.github.com/users/YosefLab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YosefLab/subscriptions","organizations_url":"https://api.github.com/users/YosefLab/orgs","repos_url":"https://api.github.com/users/YosefLab/repos","events_url":"https://api.github.com/users/YosefLab/events{/privacy}","received_events_url":"https://api.github.com/users/YosefLab/received_events","type":"Organization","site_admin":false},"repo":{"id":127217801,"name":"scVI-dev","full_name":"YosefLab/scVI-dev","owner":{"login":"YosefLab","id":13837262,"avatar_url":"https://avatars2.githubusercontent.com/u/13837262?v=4","gravatar_id":"","url":"https://api.github.com/users/YosefLab","html_url":"https://github.com/YosefLab","followers_url":"https://api.github.com/users/YosefLab/followers","following_url":"https://api.github.com/users/YosefLab/following{/other_user}","gists_url":"https://api.github.com/users/YosefLab/gists{/gist_id}","starred_url":"https://api.github.com/users/YosefLab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YosefLab/subscriptions","organizations_url":"https://api.github.com/users/YosefLab/orgs","repos_url":"https://api.github.com/users/YosefLab/repos","events_url":"https://api.github.com/users/YosefLab/events{/privacy}","received_events_url":"https://api.github.com/users/YosefLab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/YosefLab/scVI-dev","description":"Development branch of the scVI project in PyTorch","fork":false,"url":"https://api.github.com/repos/YosefLab/scVI-dev","forks_url":"https://api.github.com/repos/YosefLab/scVI-dev/forks","keys_url":"https://api.github.com/repos/YosefLab/scVI-dev/keys{/key_id}","collaborators_url":"https://api.github.com/repos/YosefLab/scVI-dev/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/YosefLab/scVI-dev/teams","hooks_url":"https://api.github.com/repos/YosefLab/scVI-dev/hooks","issue_events_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues/events{/number}","events_url":"https://api.github.com/repos/YosefLab/scVI-dev/events","assignees_url":"https://api.github.com/repos/YosefLab/scVI-dev/assignees{/user}","branches_url":"https://api.github.com/repos/YosefLab/scVI-dev/branches{/branch}","tags_url":"https://api.github.com/repos/YosefLab/scVI-dev/tags","blobs_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/refs{/sha}","trees_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/trees{/sha}","statuses_url":"https://api.github.com/repos/YosefLab/scVI-dev/statuses/{sha}","languages_url":"https://api.github.com/repos/YosefLab/scVI-dev/languages","stargazers_url":"https://api.github.com/repos/YosefLab/scVI-dev/stargazers","contributors_url":"https://api.github.com/repos/YosefLab/scVI-dev/contributors","subscribers_url":"https://api.github.com/repos/YosefLab/scVI-dev/subscribers","subscription_url":"https://api.github.com/repos/YosefLab/scVI-dev/subscription","commits_url":"https://api.github.com/repos/YosefLab/scVI-dev/commits{/sha}","git_commits_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/commits{/sha}","comments_url":"https://api.github.com/repos/YosefLab/scVI-dev/comments{/number}","issue_comment_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues/comments{/number}","contents_url":"https://api.github.com/repos/YosefLab/scVI-dev/contents/{+path}","compare_url":"https://api.github.com/repos/YosefLab/scVI-dev/compare/{base}...{head}","merges_url":"https://api.github.com/repos/YosefLab/scVI-dev/merges","archive_url":"https://api.github.com/repos/YosefLab/scVI-dev/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/YosefLab/scVI-dev/downloads","issues_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues{/number}","pulls_url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls{/number}","milestones_url":"https://api.github.com/repos/YosefLab/scVI-dev/milestones{/number}","notifications_url":"https://api.github.com/repos/YosefLab/scVI-dev/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/YosefLab/scVI-dev/labels{/name}","releases_url":"https://api.github.com/repos/YosefLab/scVI-dev/releases{/id}","deployments_url":"https://api.github.com/repos/YosefLab/scVI-dev/deployments","created_at":"2018-03-29T01:12:17Z","updated_at":"2018-05-04T20:42:00Z","pushed_at":"2018-05-04T20:42:02Z","git_url":"git://github.com/YosefLab/scVI-dev.git","ssh_url":"git@github.com:YosefLab/scVI-dev.git","clone_url":"https://github.com/YosefLab/scVI-dev.git","svn_url":"https://github.com/YosefLab/scVI-dev","homepage":null,"size":46573,"stargazers_count":10,"watchers_count":10,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":19,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit"},"forks":1,"open_issues":19,"watchers":10,"default_branch":"master"}},"base":{"label":"YosefLab:master","ref":"master","sha":"b3e07152ca9f2f3870dec298de4b94afba0f6cab","user":{"login":"YosefLab","id":13837262,"avatar_url":"https://avatars2.githubusercontent.com/u/13837262?v=4","gravatar_id":"","url":"https://api.github.com/users/YosefLab","html_url":"https://github.com/YosefLab","followers_url":"https://api.github.com/users/YosefLab/followers","following_url":"https://api.github.com/users/YosefLab/following{/other_user}","gists_url":"https://api.github.com/users/YosefLab/gists{/gist_id}","starred_url":"https://api.github.com/users/YosefLab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YosefLab/subscriptions","organizations_url":"https://api.github.com/users/YosefLab/orgs","repos_url":"https://api.github.com/users/YosefLab/repos","events_url":"https://api.github.com/users/YosefLab/events{/privacy}","received_events_url":"https://api.github.com/users/YosefLab/received_events","type":"Organization","site_admin":false},"repo":{"id":127217801,"name":"scVI-dev","full_name":"YosefLab/scVI-dev","owner":{"login":"YosefLab","id":13837262,"avatar_url":"https://avatars2.githubusercontent.com/u/13837262?v=4","gravatar_id":"","url":"https://api.github.com/users/YosefLab","html_url":"https://github.com/YosefLab","followers_url":"https://api.github.com/users/YosefLab/followers","following_url":"https://api.github.com/users/YosefLab/following{/other_user}","gists_url":"https://api.github.com/users/YosefLab/gists{/gist_id}","starred_url":"https://api.github.com/users/YosefLab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/YosefLab/subscriptions","organizations_url":"https://api.github.com/users/YosefLab/orgs","repos_url":"https://api.github.com/users/YosefLab/repos","events_url":"https://api.github.com/users/YosefLab/events{/privacy}","received_events_url":"https://api.github.com/users/YosefLab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/YosefLab/scVI-dev","description":"Development branch of the scVI project in PyTorch","fork":false,"url":"https://api.github.com/repos/YosefLab/scVI-dev","forks_url":"https://api.github.com/repos/YosefLab/scVI-dev/forks","keys_url":"https://api.github.com/repos/YosefLab/scVI-dev/keys{/key_id}","collaborators_url":"https://api.github.com/repos/YosefLab/scVI-dev/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/YosefLab/scVI-dev/teams","hooks_url":"https://api.github.com/repos/YosefLab/scVI-dev/hooks","issue_events_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues/events{/number}","events_url":"https://api.github.com/repos/YosefLab/scVI-dev/events","assignees_url":"https://api.github.com/repos/YosefLab/scVI-dev/assignees{/user}","branches_url":"https://api.github.com/repos/YosefLab/scVI-dev/branches{/branch}","tags_url":"https://api.github.com/repos/YosefLab/scVI-dev/tags","blobs_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/refs{/sha}","trees_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/trees{/sha}","statuses_url":"https://api.github.com/repos/YosefLab/scVI-dev/statuses/{sha}","languages_url":"https://api.github.com/repos/YosefLab/scVI-dev/languages","stargazers_url":"https://api.github.com/repos/YosefLab/scVI-dev/stargazers","contributors_url":"https://api.github.com/repos/YosefLab/scVI-dev/contributors","subscribers_url":"https://api.github.com/repos/YosefLab/scVI-dev/subscribers","subscription_url":"https://api.github.com/repos/YosefLab/scVI-dev/subscription","commits_url":"https://api.github.com/repos/YosefLab/scVI-dev/commits{/sha}","git_commits_url":"https://api.github.com/repos/YosefLab/scVI-dev/git/commits{/sha}","comments_url":"https://api.github.com/repos/YosefLab/scVI-dev/comments{/number}","issue_comment_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues/comments{/number}","contents_url":"https://api.github.com/repos/YosefLab/scVI-dev/contents/{+path}","compare_url":"https://api.github.com/repos/YosefLab/scVI-dev/compare/{base}...{head}","merges_url":"https://api.github.com/repos/YosefLab/scVI-dev/merges","archive_url":"https://api.github.com/repos/YosefLab/scVI-dev/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/YosefLab/scVI-dev/downloads","issues_url":"https://api.github.com/repos/YosefLab/scVI-dev/issues{/number}","pulls_url":"https://api.github.com/repos/YosefLab/scVI-dev/pulls{/number}","milestones_url":"https://api.github.com/repos/YosefLab/scVI-dev/milestones{/number}","notifications_url":"https://api.github.com/repos/YosefLab/scVI-dev/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/YosefLab/scVI-dev/labels{/name}","releases_url":"https://api.github.com/repos/YosefLab/scVI-dev/releases{/id}","deployments_url":"https://api.github.com/repos/YosefLab/scVI-dev/deployments","created_at":"2018-03-29T01:12:17Z","updated_at":"2018-05-04T20:42:00Z","pushed_at":"2018-05-04T20:42:02Z","git_url":"git://github.com/YosefLab/scVI-dev.git","ssh_url":"git@github.com:YosefLab/scVI-dev.git","clone_url":"https://github.com/YosefLab/scVI-dev.git","svn_url":"https://github.com/YosefLab/scVI-dev","homepage":null,"size":46573,"stargazers_count":10,"watchers_count":10,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"archived":false,"open_issues_count":19,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit"},"forks":1,"open_issues":19,"watchers":10,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93"},"html":{"href":"https://github.com/YosefLab/scVI-dev/pull/93"},"issue":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/issues/93"},"comments":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/issues/93/comments"},"review_comments":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93/comments"},"review_comment":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/pulls/93/commits"},"statuses":{"href":"https://api.github.com/repos/YosefLab/scVI-dev/statuses/44069abcbb34ec8acba11821957a70b81297a01c"}},"author_association":"COLLABORATOR"}}
{"actor":{"display_login":"jeff-regier"}}