{"action":"created","issue":{"url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/issues/186","repository_url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles","labels_url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/issues/186/labels{/name}","comments_url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/issues/186/comments","events_url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/issues/186/events","html_url":"https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/186","id":143283731,"number":186,"title":"Implement PhoSimPredictor class using ML results from #159","user":{"login":"jchiang87","id":1994473,"avatar_url":"https://avatars.githubusercontent.com/u/1994473?v=3","gravatar_id":"","url":"https://api.github.com/users/jchiang87","html_url":"https://github.com/jchiang87","followers_url":"https://api.github.com/users/jchiang87/followers","following_url":"https://api.github.com/users/jchiang87/following{/other_user}","gists_url":"https://api.github.com/users/jchiang87/gists{/gist_id}","starred_url":"https://api.github.com/users/jchiang87/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jchiang87/subscriptions","organizations_url":"https://api.github.com/users/jchiang87/orgs","repos_url":"https://api.github.com/users/jchiang87/repos","events_url":"https://api.github.com/users/jchiang87/events{/privacy}","received_events_url":"https://api.github.com/users/jchiang87/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/labels/PhoSim","name":"PhoSim","color":"006b75"}],"state":"open","locked":false,"assignee":{"login":"sethdigel","id":6035835,"avatar_url":"https://avatars.githubusercontent.com/u/6035835?v=3","gravatar_id":"","url":"https://api.github.com/users/sethdigel","html_url":"https://github.com/sethdigel","followers_url":"https://api.github.com/users/sethdigel/followers","following_url":"https://api.github.com/users/sethdigel/following{/other_user}","gists_url":"https://api.github.com/users/sethdigel/gists{/gist_id}","starred_url":"https://api.github.com/users/sethdigel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sethdigel/subscriptions","organizations_url":"https://api.github.com/users/sethdigel/orgs","repos_url":"https://api.github.com/users/sethdigel/repos","events_url":"https://api.github.com/users/sethdigel/events{/privacy}","received_events_url":"https://api.github.com/users/sethdigel/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/milestones/10","html_url":"https://github.com/DarkEnergyScienceCollaboration/Twinkles/milestones/Twinkles%201%20Data%20Generation%20Start","labels_url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/milestones/10/labels","id":1578719,"number":10,"title":"Twinkles 1 Data Generation Start","description":"Not sure when we'll need to start making images for reals. Beginning of August?","creator":{"login":"drphilmarshall","id":710903,"avatar_url":"https://avatars.githubusercontent.com/u/710903?v=3","gravatar_id":"","url":"https://api.github.com/users/drphilmarshall","html_url":"https://github.com/drphilmarshall","followers_url":"https://api.github.com/users/drphilmarshall/followers","following_url":"https://api.github.com/users/drphilmarshall/following{/other_user}","gists_url":"https://api.github.com/users/drphilmarshall/gists{/gist_id}","starred_url":"https://api.github.com/users/drphilmarshall/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drphilmarshall/subscriptions","organizations_url":"https://api.github.com/users/drphilmarshall/orgs","repos_url":"https://api.github.com/users/drphilmarshall/repos","events_url":"https://api.github.com/users/drphilmarshall/events{/privacy}","received_events_url":"https://api.github.com/users/drphilmarshall/received_events","type":"User","site_admin":false},"open_issues":9,"closed_issues":5,"state":"open","created_at":"2016-02-11T22:41:35Z","updated_at":"2016-03-31T02:06:43Z","due_on":"2016-08-01T07:00:00Z","closed_at":null},"comments":48,"created_at":"2016-03-24T15:50:33Z","updated_at":"2016-05-04T08:07:06Z","closed_at":null,"body":"At #159, Phil says:\r\n> The next step could be to extract the ML parts of your notebook and repackage them into a PhoSimPredictor class, which could be trained and then pickled for use before every new PhoSim run to determine which queue to use.\r\n\r\n### Status\r\n\r\n@sethdigel is investigating the long CPU time jobs in Run 1, by re-doing (with @TomGlanzman) them at 1-second exposure time (\"Run 1sx\").\r\n\r\n### Summary of Results\r\n\r\nThe Run 1 and Run 1sx CPU times are only approximately linearly related, meaning that our ability to predict the longest CPU times is very uncertain. See this [Confluence page](https://confluence.slac.stanford.edu/pages/viewpage.action?spaceKey=LSSTDESC&title=Phosim+CPU+times+in+Twinkles+Run+1+and+Run+1sx) for more details as they appear.\r\n\r\n<!-- PJM: First I copied the images from the Confluence page, via a right click in the browser. Then I pasted them in to the issue comment box, where GitHub stored and rendered them as follows:\r\n![image](https://cloud.githubusercontent.com/assets/710903/14294286/086ff634-fb25-11e5-8607-710e313a9e0d.png)\r\n![image](https://cloud.githubusercontent.com/assets/710903/14294263/feebbfee-fb24-11e5-9b60-c0c8ad01afb8.png)\r\nThen I reformatted with an html table so they appeared side by side. This took a little bit of effort - the \"preview\" feature was very useful at this step. -->\r\n\r\n<table><tr>\r\n<td><img src=\"https://cloud.githubusercontent.com/assets/710903/14294263/feebbfee-fb24-11e5-9b60-c0c8ad01afb8.png\" width=\"100%\" /></td>\r\n<td><img src=\"https://cloud.githubusercontent.com/assets/710903/14294286/086ff634-fb25-11e5-8607-710e313a9e0d.png\" width=\"100%\" /></td>\r\n</tr></table>"},"comment":{"url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/issues/comments/216775680","html_url":"https://github.com/DarkEnergyScienceCollaboration/Twinkles/issues/186#issuecomment-216775680","issue_url":"https://api.github.com/repos/DarkEnergyScienceCollaboration/Twinkles/issues/186","id":216775680,"user":{"login":"sethdigel","id":6035835,"avatar_url":"https://avatars.githubusercontent.com/u/6035835?v=3","gravatar_id":"","url":"https://api.github.com/users/sethdigel","html_url":"https://github.com/sethdigel","followers_url":"https://api.github.com/users/sethdigel/followers","following_url":"https://api.github.com/users/sethdigel/following{/other_user}","gists_url":"https://api.github.com/users/sethdigel/gists{/gist_id}","starred_url":"https://api.github.com/users/sethdigel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sethdigel/subscriptions","organizations_url":"https://api.github.com/users/sethdigel/orgs","repos_url":"https://api.github.com/users/sethdigel/repos","events_url":"https://api.github.com/users/sethdigel/events{/privacy}","received_events_url":"https://api.github.com/users/sethdigel/received_events","type":"User","site_admin":false},"created_at":"2016-05-04T08:07:06Z","updated_at":"2016-05-04T08:07:06Z","body":"I've made a branch called issue/186/cpu_time_predictor and pushed three files:\r\ndata/run1_metadata_v6.csv, which has the complete metadata for the Run 1 phosim visits, including CPU times scaled to their equivalents on 'fell' batch hosts, and estimated CPU times from Run 1sx (1-s visits) for the 103 Run 1 visits that reached the 120 hr CPU limit.\r\n\r\npython/desc/twinkles/run1_cpu_generate_rf.py, which generates a Random Forest predictor based on the moonalt, moonphase, and filter metadata.  The predictor is saved as a pickle file (34 Mbyte for 3 features).  A fixed seed is specified in the code, so the same RF is generated each time.  For the particular seed I picked (999) th MSE (in log10 CPU time) is 0.0047.\r\n\r\npython/desc/twinkles/phosim_cpu_pred.py, which implements a class, possibly stupid and inefficient, that applies the RF predictor to estimate CPU times for 30-s phosim visits based on user-specified moonalt, moonphase, and filter.  Alternatively, an obsHistID from the kracken_1042 Opsim run can be specified and the metadata will be looked up in the corresponding sqlite database.  (Right now the code is hardwired to look for the database in a certain directory on SLAC Linux.)  The code has a couple of simple examples, including one that generates predicted fell CPU times for each of the Run 1 visits and compares them with the actual times:\r\n\r\n![predicted_vs_actual](https://cloud.githubusercontent.com/assets/6035835/15008287/14373d62-1194-11e6-8c5a-ad72e4de2d1d.png)\r\n"}}