{"action":"created","issue":{"url":"https://api.github.com/repos/desihub/desitarget/issues/66","repository_url":"https://api.github.com/repos/desihub/desitarget","labels_url":"https://api.github.com/repos/desihub/desitarget/issues/66/labels{/name}","comments_url":"https://api.github.com/repos/desihub/desitarget/issues/66/comments","events_url":"https://api.github.com/repos/desihub/desitarget/issues/66/events","html_url":"https://github.com/desihub/desitarget/pull/66","id":177310258,"number":66,"title":"Add bdt4 qso","user":{"login":"Cyeche","id":22040400,"avatar_url":"https://avatars.githubusercontent.com/u/22040400?v=3","gravatar_id":"","url":"https://api.github.com/users/Cyeche","html_url":"https://github.com/Cyeche","followers_url":"https://api.github.com/users/Cyeche/followers","following_url":"https://api.github.com/users/Cyeche/following{/other_user}","gists_url":"https://api.github.com/users/Cyeche/gists{/gist_id}","starred_url":"https://api.github.com/users/Cyeche/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Cyeche/subscriptions","organizations_url":"https://api.github.com/users/Cyeche/orgs","repos_url":"https://api.github.com/users/Cyeche/repos","events_url":"https://api.github.com/users/Cyeche/events{/privacy}","received_events_url":"https://api.github.com/users/Cyeche/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"moustakas","id":1431820,"avatar_url":"https://avatars.githubusercontent.com/u/1431820?v=3","gravatar_id":"","url":"https://api.github.com/users/moustakas","html_url":"https://github.com/moustakas","followers_url":"https://api.github.com/users/moustakas/followers","following_url":"https://api.github.com/users/moustakas/following{/other_user}","gists_url":"https://api.github.com/users/moustakas/gists{/gist_id}","starred_url":"https://api.github.com/users/moustakas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moustakas/subscriptions","organizations_url":"https://api.github.com/users/moustakas/orgs","repos_url":"https://api.github.com/users/moustakas/repos","events_url":"https://api.github.com/users/moustakas/events{/privacy}","received_events_url":"https://api.github.com/users/moustakas/received_events","type":"User","site_admin":false},"assignees":[{"login":"moustakas","id":1431820,"avatar_url":"https://avatars.githubusercontent.com/u/1431820?v=3","gravatar_id":"","url":"https://api.github.com/users/moustakas","html_url":"https://github.com/moustakas","followers_url":"https://api.github.com/users/moustakas/followers","following_url":"https://api.github.com/users/moustakas/following{/other_user}","gists_url":"https://api.github.com/users/moustakas/gists{/gist_id}","starred_url":"https://api.github.com/users/moustakas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/moustakas/subscriptions","organizations_url":"https://api.github.com/users/moustakas/orgs","repos_url":"https://api.github.com/users/moustakas/repos","events_url":"https://api.github.com/users/moustakas/events{/privacy}","received_events_url":"https://api.github.com/users/moustakas/received_events","type":"User","site_admin":false},{"login":"geordie666","id":13952581,"avatar_url":"https://avatars.githubusercontent.com/u/13952581?v=3","gravatar_id":"","url":"https://api.github.com/users/geordie666","html_url":"https://github.com/geordie666","followers_url":"https://api.github.com/users/geordie666/followers","following_url":"https://api.github.com/users/geordie666/following{/other_user}","gists_url":"https://api.github.com/users/geordie666/gists{/gist_id}","starred_url":"https://api.github.com/users/geordie666/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/geordie666/subscriptions","organizations_url":"https://api.github.com/users/geordie666/orgs","repos_url":"https://api.github.com/users/geordie666/repos","events_url":"https://api.github.com/users/geordie666/events{/privacy}","received_events_url":"https://api.github.com/users/geordie666/received_events","type":"User","site_admin":false},{"login":"nathalieGit","id":16261908,"avatar_url":"https://avatars.githubusercontent.com/u/16261908?v=3","gravatar_id":"","url":"https://api.github.com/users/nathalieGit","html_url":"https://github.com/nathalieGit","followers_url":"https://api.github.com/users/nathalieGit/followers","following_url":"https://api.github.com/users/nathalieGit/following{/other_user}","gists_url":"https://api.github.com/users/nathalieGit/gists{/gist_id}","starred_url":"https://api.github.com/users/nathalieGit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathalieGit/subscriptions","organizations_url":"https://api.github.com/users/nathalieGit/orgs","repos_url":"https://api.github.com/users/nathalieGit/repos","events_url":"https://api.github.com/users/nathalieGit/events{/privacy}","received_events_url":"https://api.github.com/users/nathalieGit/received_events","type":"User","site_admin":false}],"milestone":null,"comments":38,"created_at":"2016-09-15T22:56:27Z","updated_at":"2016-09-27T00:09:46Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/desihub/desitarget/pulls/66","html_url":"https://github.com/desihub/desitarget/pull/66","diff_url":"https://github.com/desihub/desitarget/pull/66.diff","patch_url":"https://github.com/desihub/desitarget/pull/66.patch"},"body":"Context:\r\nThe BDT selection has been implemented in the cuts.py\r\nselect_targets  module was run over stripe 82  with a preliminary version of dr3 sweep. The results (see attached plots) look as they were expected: an improvement with BDT for faint and high-z QSOs.\r\nIt was run over whole dr2 sweep without problem. The target density for EDR region for BDT is ~260 deg^-2.\r\n\r\nTechnical information:\r\n- The new version of cuts.py required the scikit-learn package which is by default in anaconda but not in the \"mini\" anaconda used by DESI\r\n- An argument is added in select_targets module allowing to switch from color cut selection to BDT selection. The default is BDT selection now.\r\n\r\nChristophe. \r\n\r\n![completness_desi_target_dr3_vs_redshift](https://cloud.githubusercontent.com/assets/22040400/18570046/b6711afe-7b5b-11e6-8f3e-38657e489288.gif)\r\n![completness_desi_target_dr3_vs_rmag](https://cloud.githubusercontent.com/assets/22040400/18570055/c3f6ce30-7b5b-11e6-99ef-3ed515d8ff63.gif)"},"comment":{"url":"https://api.github.com/repos/desihub/desitarget/issues/comments/249733076","html_url":"https://github.com/desihub/desitarget/pull/66#issuecomment-249733076","issue_url":"https://api.github.com/repos/desihub/desitarget/issues/66","id":249733076,"user":{"login":"sbailey","id":218471,"avatar_url":"https://avatars.githubusercontent.com/u/218471?v=3","gravatar_id":"","url":"https://api.github.com/users/sbailey","html_url":"https://github.com/sbailey","followers_url":"https://api.github.com/users/sbailey/followers","following_url":"https://api.github.com/users/sbailey/following{/other_user}","gists_url":"https://api.github.com/users/sbailey/gists{/gist_id}","starred_url":"https://api.github.com/users/sbailey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sbailey/subscriptions","organizations_url":"https://api.github.com/users/sbailey/orgs","repos_url":"https://api.github.com/users/sbailey/repos","events_url":"https://api.github.com/users/sbailey/events{/privacy}","received_events_url":"https://api.github.com/users/sbailey/received_events","type":"User","site_admin":false},"created_at":"2016-09-27T00:09:46Z","updated_at":"2016-09-27T00:09:46Z","body":"Documenting an in-person conversation with Christophe:\r\n\r\nBefore merging this PR:\r\n  * [ ] give @weaverba137 a few days to explore/implement/recommend alternate I/O models; otherwise we'll proceed with the compressed tarball version of the existing files and save the refactoring of the I/O for when scikit-learn actually makes a change that breaks it or when we actually need to read these files from outside of python\r\n  * [ ] rename \"bdt\" to \"randomforest\" or similar, since this isn't a BDT.  Maybe there will be additional QSOSelection options in the future including \"adaboost\" or \"bdt\" or \"neuralnetwork\" or ...\r\n  * [ ] Add what QSOSelection option was used to the output file header.  I haven't given up on having a small number of QSO targeting bits for color cuts vs. random forests, but in the meantime at minimum the file must contain the provenance of what method was used.\r\n\r\nShort term, but not directly blocking for this PR:\r\n  * [ ] organize the training and validation files and put those on NERSC; ask @weaverba137 for where they best go and how to back them up.\r\n  * [ ] provide code in `desitarget/py/desitarget/training/randomforest.py` (or similar) that converts the training data into the files used in `desitarget/py/desitarget/data/bdt_model_dr3` (which will also be renamed to not \"bdt\").\r\n  * [ ] provide a technote or other documentation about how to use that training code\r\n  * [ ] figure out how to set random number seeds for scikit-learn so that the training is reproducible\r\n"}}
{"actor":{"display_login":"sbailey"}}