{"action":"created","issue":{"url":"https://api.github.com/repos/StrikeNP/trac_test/issues/697","repository_url":"https://api.github.com/repos/StrikeNP/trac_test","labels_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/697/labels{/name}","comments_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/697/comments","events_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/697/events","html_url":"https://github.com/StrikeNP/trac_test/issues/697","id":330468184,"node_id":"MDU6SXNzdWUzMzA0NjgxODQ=","number":697,"title":"What causes noise in SILHS? (Trac #697)","user":{"login":"StrikeNP","id":2577059,"node_id":"MDQ6VXNlcjI1NzcwNTk=","avatar_url":"https://avatars0.githubusercontent.com/u/2577059?v=4","gravatar_id":"","url":"https://api.github.com/users/StrikeNP","html_url":"https://github.com/StrikeNP","followers_url":"https://api.github.com/users/StrikeNP/followers","following_url":"https://api.github.com/users/StrikeNP/following{/other_user}","gists_url":"https://api.github.com/users/StrikeNP/gists{/gist_id}","starred_url":"https://api.github.com/users/StrikeNP/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrikeNP/subscriptions","organizations_url":"https://api.github.com/users/StrikeNP/orgs","repos_url":"https://api.github.com/users/StrikeNP/repos","events_url":"https://api.github.com/users/StrikeNP/events{/privacy}","received_events_url":"https://api.github.com/users/StrikeNP/received_events","type":"User","site_admin":false},"labels":[{"id":958019852,"node_id":"MDU6TGFiZWw5NTgwMTk4NTI=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/Migrated%20from%20Trac","name":"Migrated from Trac","color":"FFFFFF","default":false},{"id":958042830,"node_id":"MDU6TGFiZWw5NTgwNDI4MzA=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/clubb_src","name":"clubb_src","color":"FFFFFF","default":false},{"id":958019855,"node_id":"MDU6TGFiZWw5NTgwMTk4NTU=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/defect","name":"defect","color":"FFFFFF","default":false},{"id":958114258,"node_id":"MDU6TGFiZWw5NTgxMTQyNTg=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/raut@uwm.edu","name":"raut@uwm.edu","color":"FFFFFF","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/StrikeNP/trac_test/milestones/8","html_url":"https://github.com/StrikeNP/trac_test/milestone/8","labels_url":"https://api.github.com/repos/StrikeNP/trac_test/milestones/8/labels","id":3408876,"node_id":"MDk6TWlsZXN0b25lMzQwODg3Ng==","number":8,"title":"Improve SILHS","description":null,"creator":{"login":"StrikeNP","id":2577059,"node_id":"MDQ6VXNlcjI1NzcwNTk=","avatar_url":"https://avatars0.githubusercontent.com/u/2577059?v=4","gravatar_id":"","url":"https://api.github.com/users/StrikeNP","html_url":"https://github.com/StrikeNP","followers_url":"https://api.github.com/users/StrikeNP/followers","following_url":"https://api.github.com/users/StrikeNP/following{/other_user}","gists_url":"https://api.github.com/users/StrikeNP/gists{/gist_id}","starred_url":"https://api.github.com/users/StrikeNP/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrikeNP/subscriptions","organizations_url":"https://api.github.com/users/StrikeNP/orgs","repos_url":"https://api.github.com/users/StrikeNP/repos","events_url":"https://api.github.com/users/StrikeNP/events{/privacy}","received_events_url":"https://api.github.com/users/StrikeNP/received_events","type":"User","site_admin":false},"open_issues":32,"closed_issues":49,"state":"open","created_at":"2018-06-07T19:59:06Z","updated_at":"2018-06-09T22:14:51Z","due_on":null,"closed_at":null},"comments":20,"created_at":"2018-06-07T23:18:21Z","updated_at":"2018-06-09T23:41:49Z","closed_at":null,"author_association":"OWNER","body":"In ticket:682, we are attempting to use mirrored sampling and importance sampling to reduce noise in SILHS.\n\nI am starting this ticket just to record some general observations regarding noise in SILHS.\n\nAttachments:\n\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrainm_noise.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrainm_noise.diff\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrainm_T226.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrainm_T226_noprecipfrac.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/mu_rr_n_T226.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/mixt_frac_k_lh_start.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/dp1_stratified_scatter.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/mixt_frac_stratified_comp.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_noise.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_noise_noprecip.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/chi_rr_scatter.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/chi_rr_scatter_100000.png\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/cloud_precip_weighting.maff\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/scatter_cloud_precip.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_new_code_noise_comp.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_precip_only.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_auto.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/lh_mixt_frac_unweighted.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/cloud_frac1_2.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_auto_mixture_comp_importance.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_mf_sampling.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/precip_frac_unweighted_mf_importance.svg\nhttp://carson.math.uwm.edu/trac/clubb/attachment/ticket/697/rrm_precip_cloud_only.svg\n\nMigrated from http://carson.math.uwm.edu/trac/clubb/ticket/697\n```json\n{\n    \"status\": \"closed\", \n    \"changetime\": \"2014-09-01T17:36:30\", \n    \"description\": \"In ticket:682, we are attempting to use mirrored sampling and importance sampling to reduce noise in SILHS.\\n\\nI am starting this ticket just to record some general observations regarding noise in SILHS.\", \n    \"reporter\": \"raut@uwm.edu\", \n    \"cc\": \"vlarson@uwm.edu\", \n    \"resolution\": \"continued\", \n    \"_ts\": \"1409592990461502\", \n    \"component\": \"clubb_src\", \n    \"summary\": \"What causes noise in SILHS?\", \n    \"priority\": \"minor\", \n    \"keywords\": \"silhs noise\", \n    \"time\": \"2014-06-11T00:23:50\", \n    \"milestone\": \"Improve SILHS\", \n    \"owner\": \"raut@uwm.edu\", \n    \"type\": \"defect\"\n}\n```\n"},"comment":{"url":"https://api.github.com/repos/StrikeNP/trac_test/issues/comments/396006622","html_url":"https://github.com/StrikeNP/trac_test/issues/697#issuecomment-396006622","issue_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/697","id":396006622,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NjAwNjYyMg==","user":{"login":"StrikeNP","id":2577059,"node_id":"MDQ6VXNlcjI1NzcwNTk=","avatar_url":"https://avatars0.githubusercontent.com/u/2577059?v=4","gravatar_id":"","url":"https://api.github.com/users/StrikeNP","html_url":"https://github.com/StrikeNP","followers_url":"https://api.github.com/users/StrikeNP/followers","following_url":"https://api.github.com/users/StrikeNP/following{/other_user}","gists_url":"https://api.github.com/users/StrikeNP/gists{/gist_id}","starred_url":"https://api.github.com/users/StrikeNP/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrikeNP/subscriptions","organizations_url":"https://api.github.com/users/StrikeNP/orgs","repos_url":"https://api.github.com/users/StrikeNP/repos","events_url":"https://api.github.com/users/StrikeNP/events{/privacy}","received_events_url":"https://api.github.com/users/StrikeNP/received_events","type":"User","site_admin":false},"created_at":"2018-06-09T23:41:49Z","updated_at":"2018-06-09T23:41:49Z","author_association":"OWNER","body":"Trac update at 2014/08/09 at 03:29\n\n\n* raut@uwm.edu changed _comment0 from:\n\n> I kind of broke the SILHS noise problem into two parts: noise inherent in the cloud weighted sampling process, and noise that comes from the vertical overlap code. Today, I investigated the noise in the cloud weighted sampling process.\n> \n> I wrote a Canopy script (awesome program, by the way) that plots timeseries of two variables (an analytic variable and the equivalent SILHS variable) at `k_lh_start`. It uses the `k_lh_start` output added in r7181 to pick what height level to plot at. My goal was to make it easy to assess the amount of noise that is occurring at `k_lh_start` and only `k_lh_start`.\n> \n> I quickly realized that `mixt_frac` was very noisy when compared with `lh_mixt_frac`. I decided to test whether this was reasonable. I wrote some quick code in Canopy to estimate a \"quick and dirty\" `mixt_frac`. It looked something like this:\n> \n> {{{\n> num_samples = 100\n> count = 0\n> for i in range(0,num_samples):\n>     if ( random.random() < mixt_frac ):\n>         count = count + 1\n> mixt_frac_est = count / num_samples\n> }}}\n> \n> In attachment:mixt_frac_k_lh_start.svg, I plotted timeseries of three lines (all at `k_lh_start`). The analytic `mixt_frac` is shown in blue. The SILHS-predicted `mixt_frac` is shown in green. Finally, my quick and dirty `mixt_frac` estimate is shown in red. Unfortunately, both the SILHS estimate of `mixt_frac` and my calculation seem to show the same amount of noise. I guess that we can't get much better than that with 100 sample points. From this, it seems that the SILHS answer is reasonable.\n> \n> But wait... there's more!! We're supposed to be stratifying in SILHS, right? Looking at the algorithm above, imagine what stratification would do. We'd be guaranteed that the fraction of sample points that satisfy `rnd < mixt_frac` will either be equal to `mixt_frac` or else off by just 0.01. The sampling of `mixt_frac` would be much, much less noisy.\n> \n> So, why don't we see that behavior in the SILHS line? Well, I took a look in `latin_hypercube_driver_module.F90` and made a very interesting discovery: when `l_lh_cloud_weighted_sampling = .true.`, we don't stratify the `d+1` element!! The relevant line in code is [browser:CLUBB/trunk/src/SILHS/latin_hypercube_driver_module.F90?rev=7182#L1332 line 1332 of latin_hypercube_driver_module.F90] (and lines 1337, 1365, and 1370, but they're all essentially the same). We work so hard to produce a height-time matrix and all that fanciness, just to throw it all away (for the `d+1` element, at least)!\n> \n> So, the next thing I want to do is definitely try stratifying the `d+1` element. I'll have to think about how to do that. We already pass in a `p_matrix_element` to the subroutine to stratify the `chi` uniform element, but I wouldn't use that, because we'd have an undesired correlation between `chi` and the `d+1` element. We could pass in another `p_matrix_element` that comes from the `d+1` row of the height time matrix and use that. That would solve the correlation problem. However, I'm not particularly excited about that option because it wouldn't stratify each case completely. There are four different cases where the variate is assigned a value (the four lines mentioned in the previous paragraph), and it would be awesome if we could stratify all four cases individually. I'll come up with an algorithm to do this.\n> \n> Anyway, I think this is going to finally solve all of our SILHS problems!\n\nto:\n\n> 1407555032899977\n\n\n* raut@uwm.edu commented:\n\nI kind of broke the SILHS noise problem into two parts: noise inherent in the cloud weighted sampling process, and noise that comes from the vertical overlap code. Today, I investigated the noise in the cloud weighted sampling process.\n\nI wrote a Canopy script (awesome program, by the way) that plots timeseries of two variables (an analytic variable and the equivalent SILHS variable) at `k_lh_start`. It uses the `k_lh_start` output added in r7181 to pick what height level to plot at. My goal was to make it easy to assess the amount of noise that is occurring at `k_lh_start` and only `k_lh_start`.\n\nI quickly realized that `mixt_frac` was very noisy when compared with `lh_mixt_frac`. I decided to test whether this was reasonable. I wrote some quick code in Canopy to estimate a \"quick and dirty\" `mixt_frac`. It looked something like this:\n\n\n```text\nnum_samples = 100\ncount = 0\nfor i in range(0,num_samples):\n    if ( random.random() < mixt_frac ):\n        count = count + 1\nmixt_frac_est = count / num_samples\n```\n\nIn attachment:mixt_frac_k_lh_start.svg, I plotted timeseries of three lines (all at `k_lh_start`). The analytic `mixt_frac` is shown in blue. The SILHS-predicted `mixt_frac` is shown in green. Finally, my quick and dirty `mixt_frac` estimate is shown in red. Unfortunately, both the SILHS estimate of `mixt_frac` and my calculation seem to show the same amount of noise. I guess that we can't get much better than that with 100 sample points. From this, it seems that the SILHS answer is reasonable.\n\nBut wait... there's more!! We're supposed to be stratifying in SILHS, right? Looking at the algorithm above, imagine what stratification would do. We'd be guaranteed that the fraction of sample points that satisfy `rnd < mixt_frac` will either be equal to `mixt_frac` or else off by just 0.01. The sampling of `mixt_frac` would be much, much less noisy.\n\nSo, why don't we see that behavior in the SILHS line? Well, I took a look in `latin_hypercube_driver_module.F90` and made a very interesting discovery: when `l_lh_cloud_weighted_sampling = .true.`, we don't stratify the `d+1` element!! The relevant line in code is [browser:CLUBB/trunk/src/SILHS/latin_hypercube_driver_module.F90?rev=7182#L1332 line 1332 of latin_hypercube_driver_module.F90] (and lines 1337, 1365, and 1370, but they're all essentially the same). We work so hard to produce a height-time matrix and all that fanciness, just to throw it all away (for the `d+1` element, at least)!\n\nSo, the next thing I want to do is definitely try stratifying the `d+1` element. I'll have to think about how to do that. We already pass in a `p_matrix_element` to the subroutine to stratify the `chi` uniform element, but I wouldn't use that, because we'd have an undesired correlation between `chi` and the `d+1` element. We could pass in another `p_matrix_element` that comes from the `d+1` row of the height time matrix and use that. That would solve the correlation problem. However, I'm not particularly excited about that option because it wouldn't stratify each case completely. There are four different cases where the variate is assigned a value (the four lines mentioned in the previous paragraph), and it would be awesome if we could stratify all four cases individually (and completely). I'll come up with an algorithm to do this.\n\nAnyway, I think this is going to finally solve all of our SILHS problems!\n\n"}}
{"actor":{"display_login":"StrikeNP"}}