{"action":"created","issue":{"url":"https://api.github.com/repos/StrikeNP/trac_test/issues/777","repository_url":"https://api.github.com/repos/StrikeNP/trac_test","labels_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/777/labels{/name}","comments_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/777/comments","events_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/777/events","html_url":"https://github.com/StrikeNP/trac_test/issues/777","id":332200429,"node_id":"MDU6SXNzdWUzMzIyMDA0Mjk=","number":777,"title":"Modify ADG2 to control the sprawling standard deviation of chi in PDF component 1 (Trac #777)","user":{"login":"StrikeNP","id":2577059,"node_id":"MDQ6VXNlcjI1NzcwNTk=","avatar_url":"https://avatars0.githubusercontent.com/u/2577059?v=4","gravatar_id":"","url":"https://api.github.com/users/StrikeNP","html_url":"https://github.com/StrikeNP","followers_url":"https://api.github.com/users/StrikeNP/followers","following_url":"https://api.github.com/users/StrikeNP/following{/other_user}","gists_url":"https://api.github.com/users/StrikeNP/gists{/gist_id}","starred_url":"https://api.github.com/users/StrikeNP/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrikeNP/subscriptions","organizations_url":"https://api.github.com/users/StrikeNP/orgs","repos_url":"https://api.github.com/users/StrikeNP/repos","events_url":"https://api.github.com/users/StrikeNP/events{/privacy}","received_events_url":"https://api.github.com/users/StrikeNP/received_events","type":"User","site_admin":false},"labels":[{"id":963997231,"node_id":"MDU6TGFiZWw5NjM5OTcyMzE=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/Migrated%20from%20Trac","name":"Migrated from Trac","color":"FFFFFF","default":false},{"id":963998173,"node_id":"MDU6TGFiZWw5NjM5OTgxNzM=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/bmg2@uwm.edu","name":"bmg2@uwm.edu","color":"FFFFFF","default":false},{"id":963997516,"node_id":"MDU6TGFiZWw5NjM5OTc1MTY=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/clubb_src","name":"clubb_src","color":"FFFFFF","default":false},{"id":963997043,"node_id":"MDU6TGFiZWw5NjM5OTcwNDM=","url":"https://api.github.com/repos/StrikeNP/trac_test/labels/enhancement","name":"enhancement","color":"a2eeef","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/StrikeNP/trac_test/milestones/26","html_url":"https://github.com/StrikeNP/trac_test/milestone/26","labels_url":"https://api.github.com/repos/StrikeNP/trac_test/milestones/26/labels","id":3423502,"node_id":"MDk6TWlsZXN0b25lMzQyMzUwMg==","number":26,"title":"Create and implement a new trivariate analytic double Gaussian PDF in CLUBB","description":null,"creator":{"login":"StrikeNP","id":2577059,"node_id":"MDQ6VXNlcjI1NzcwNTk=","avatar_url":"https://avatars0.githubusercontent.com/u/2577059?v=4","gravatar_id":"","url":"https://api.github.com/users/StrikeNP","html_url":"https://github.com/StrikeNP","followers_url":"https://api.github.com/users/StrikeNP/followers","following_url":"https://api.github.com/users/StrikeNP/following{/other_user}","gists_url":"https://api.github.com/users/StrikeNP/gists{/gist_id}","starred_url":"https://api.github.com/users/StrikeNP/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrikeNP/subscriptions","organizations_url":"https://api.github.com/users/StrikeNP/orgs","repos_url":"https://api.github.com/users/StrikeNP/repos","events_url":"https://api.github.com/users/StrikeNP/events{/privacy}","received_events_url":"https://api.github.com/users/StrikeNP/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":0,"state":"open","created_at":"2018-06-13T23:27:18Z","updated_at":"2018-06-13T23:31:00Z","due_on":null,"closed_at":null},"comments":76,"created_at":"2018-06-13T23:27:49Z","updated_at":"2018-06-16T14:09:19Z","closed_at":null,"author_association":"OWNER","body":"The methods that have been used to close the equation set for calculating PDF component means and standard deviations in CLUBB's two-component PDF have all resulted in a huge standard deviation of chi in the 1st PDF component, especially when mixture fraction is small (skewness is high).  This has been a problem with ADG1, ADG2, and 3-D Luhar.  The huge 1st component standard deviations don't produce problems in the calculation of cloud fraction or mean cloud water mixing ratio.  However, when chi is distributed jointly with a hydrometeor field, problems show up in the joint PDF.  The huge standard deviation of chi in the 1st component causes problems with some microphysical process rates.  Over the past year, I've had problems with over-evaporation in RICO at high altitudes because of this issue.  This happens because the huge ```stdev_chi_1``` causes [attachment:RICO_joint_pdf_chi_rr_4200min_2780m.https://github.com/larson-group/png/issues/735 too many large values of rain water mixing ratio to be placed in air that is too dry].\n\nA couple of days ago (papers:https://github.com/larson-group/clubb/issues/7), I looked at results of frozen hydrometeor process rates.  Most of them looked horrible.  When I looked at the profiles of back-solved correlations between chi and hydrometeor fields, they had to be truncated at 1 in the problem regions because the values would have been unrealizable.  In short, I think the problem of having a very broad PDF component 1 of chi when mixture fraction is small is producing these problems with frozen hydrometeor fields.\n\nAgain, looking at attachment:RICO_joint_pdf_chi_rr_4200min_2780m.https://github.com/larson-group/png/issues/735​, CLUBB's PDF of chi in component 1 is too broad.  CLUBB is accounting for the variance in chi at this altitude by increasing the component variances rather than spreading the component means.  This is the root of the problem.  CLUBB needs to increase the spread of the means and decrease the component variances in this situation.  In that scatterplot, CLUBB has a high correlation of rr and chi within the components.  Looking at the SAM scatterplot, CLUBB would be better off by spreading the component means of chi further apart and then having a within-component correlation of almost 0.  In that scatterplot, not only do CLUBB chi values extend too far to the left (dry), but also too far to the right (moist).  I think this is wreaking havoc with frozen hydrometeor fields.  For example, this \"too moist in cloud\" region would cause an excessive collection of cloud water by graupel (Morrison's ```PSACWG```).\n\nMy goal is to develop a new way to close the equation set for calculating PDF component means and standard deviations that controls the sprawling ```stdev_chi_1```.  Now each PDF component's standard deviation of chi is simply calculated as part of a change of coordinates.  The relationship is simple -- when the component standard deviations of rt and theta-l decrease, so does the component standard deviation of chi.  The new equation set would be designed to control the component standard deviations of rt and theta-l.\n\nI don't want to completely reinvent the wheel; I only want to partially reinvent it.  I would like to modify the current ADG2 PDF.  Particularly, I would still use the current ADG2 to produce the mixture fractions and the PDF parameters for vertical velocity.  I prefer ADG2 to ADG1 because it produces larger mixture fractions (which is beneficial in the production of the hydrometeor marginal PDFs as well as in the microphysics process rates) and it produces a much better PDF of vertical velocity.  Additionally, using ADG2 to produce the mixture fractions means that the hydrometeor PDF plots and the goodness-of-fit test results won't be altered at all.  However, I would handle the calculation of the PDF parameters for rt and theta-l in a new manner.  I have three ideas for the calculation of these parameters, which I will detail in the comments below.\n\n\nAttachments:\n\n[plot_mc_var_covar_1.maff](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/plot_mc_var_covar_1.maff)\n[rt_thl_PDF_xm_xp2_xp3.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/rt_thl_PDF_xm_xp2_xp3.pdf)\n[rt_thl_PDF_xm_xp2_new.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/rt_thl_PDF_xm_xp2_new.pdf)\n[analytic_pdf_formulation_20160628.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/analytic_pdf_formulation_20160628.pdf)\n[test_PDF_w_Fw_0p5_zetaw_0.png](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/test_PDF_w_Fw_0p5_zetaw_0.png)\n[test_PDF_w_Fw_0p5_zetaw_3.png](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/test_PDF_w_Fw_0p5_zetaw_3.png)\n[test_PDF_w_Fw_0p5_zetaw_m0p25.png](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/test_PDF_w_Fw_0p5_zetaw_m0p25.png)\n[plot_compare_w_Fw_0p5_zetaw_0.png](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/plot_compare_w_Fw_0p5_zetaw_0.png)\n[plot_compare_w_Fw_Skw_20_zetaw_0.png](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/plot_compare_w_Fw_Skw_20_zetaw_0.png)\n[new_PDF_w_or_setting_variable.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/new_PDF_w_or_setting_variable.pdf)\n[new_PDF_rt_thl_or_responding_variables.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/new_PDF_rt_thl_or_responding_variables.pdf)\n[check_pos_at_root.m](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/check_pos_at_root.m)\n[x1ap_x2bp_bivar_NN_PDF_integral.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/x1ap_x2bp_bivar_NN_PDF_integral.pdf)\n[x1_alpha_p_univar_N_PDF_integral.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/x1_alpha_p_univar_N_PDF_integral.pdf)\n[wp4_semi_implicit_and_discretization.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/wp4_semi_implicit_and_discretization.pdf)\n[wpxp2_implicit_and_discretization.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/wpxp2_implicit_and_discretization.pdf)\n[wp2xp_semi_implicit_and_discretization.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/wp2xp_semi_implicit_and_discretization.pdf)\n[adg3_pdf_eqns.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/adg3_pdf_eqns.pdf)\n[adg3_pdf_eqns_realizability.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/adg3_pdf_eqns_realizability.pdf)\n[new_pdf_use_sgn_Skw_sgn_Skx.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/new_pdf_use_sgn_Skw_sgn_Skx.pdf)\n[wap_rcbp_bivar_NN_PDF_integral.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/wap_rcbp_bivar_NN_PDF_integral.pdf)\n[rtprcp_thlprcp_rcp2_PDF_integrals.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/rtprcp_thlprcp_rcp2_PDF_integrals.pdf)\n[wpxpyp_semi_implicit_and_discretization.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/wpxpyp_semi_implicit_and_discretization.pdf)\n[x1ap_x2bp_x3cp_trivar_NNN_PDF_integral.pdf](https://github.com/larson-group/trac_attachment_archive/blob/master/clubb/x1ap_x2bp_x3cp_trivar_NNN_PDF_integral.pdf)\n\nMigrated from http://carson.math.uwm.edu/trac/clubb/ticket/777\n```json\n{\n    \"status\": \"new\", \n    \"changetime\": \"2018-05-21T20:19:25\", \n    \"description\": \"The methods that have been used to close the equation set for calculating PDF component means and standard deviations in CLUBB's two-component PDF have all resulted in a huge standard deviation of chi in the 1st PDF component, especially when mixture fraction is small (skewness is high).  This has been a problem with ADG1, ADG2, and 3-D Luhar.  The huge 1st component standard deviations don't produce problems in the calculation of cloud fraction or mean cloud water mixing ratio.  However, when chi is distributed jointly with a hydrometeor field, problems show up in the joint PDF.  The huge standard deviation of chi in the 1st component causes problems with some microphysical process rates.  Over the past year, I've had problems with over-evaporation in RICO at high altitudes because of this issue.  This happens because the huge {{{stdev_chi_1}}} causes [attachment:RICO_joint_pdf_chi_rr_4200min_2780m.png:ticket:735 too many large values of rain water mixing ratio to be placed in air that is too dry].\\n\\nA couple of days ago (papers:comment:152:ticket:7), I looked at results of frozen hydrometeor process rates.  Most of them looked horrible.  When I looked at the profiles of back-solved correlations between chi and hydrometeor fields, they had to be truncated at 1 in the problem regions because the values would have been unrealizable.  In short, I think the problem of having a very broad PDF component 1 of chi when mixture fraction is small is producing these problems with frozen hydrometeor fields.\\n\\nAgain, looking at attachment:RICO_joint_pdf_chi_rr_4200min_2780m.png:ticket:735\\u200b, CLUBB's PDF of chi in component 1 is too broad.  CLUBB is accounting for the variance in chi at this altitude by increasing the component variances rather than spreading the component means.  This is the root of the problem.  CLUBB needs to increase the spread of the means and decrease the component variances in this situation.  In that scatterplot, CLUBB has a high correlation of rr and chi within the components.  Looking at the SAM scatterplot, CLUBB would be better off by spreading the component means of chi further apart and then having a within-component correlation of almost 0.  In that scatterplot, not only do CLUBB chi values extend too far to the left (dry), but also too far to the right (moist).  I think this is wreaking havoc with frozen hydrometeor fields.  For example, this \\\"too moist in cloud\\\" region would cause an excessive collection of cloud water by graupel (Morrison's {{{PSACWG}}}).\\n\\nMy goal is to develop a new way to close the equation set for calculating PDF component means and standard deviations that controls the sprawling {{{stdev_chi_1}}}.  Now each PDF component's standard deviation of chi is simply calculated as part of a change of coordinates.  The relationship is simple -- when the component standard deviations of rt and theta-l decrease, so does the component standard deviation of chi.  The new equation set would be designed to control the component standard deviations of rt and theta-l.\\n\\nI don't want to completely reinvent the wheel; I only want to partially reinvent it.  I would like to modify the current ADG2 PDF.  Particularly, I would still use the current ADG2 to produce the mixture fractions and the PDF parameters for vertical velocity.  I prefer ADG2 to ADG1 because it produces larger mixture fractions (which is beneficial in the production of the hydrometeor marginal PDFs as well as in the microphysics process rates) and it produces a much better PDF of vertical velocity.  Additionally, using ADG2 to produce the mixture fractions means that the hydrometeor PDF plots and the goodness-of-fit test results won't be altered at all.  However, I would handle the calculation of the PDF parameters for rt and theta-l in a new manner.  I have three ideas for the calculation of these parameters, which I will detail in the comments below.\\n\", \n    \"reporter\": \"bmg2@uwm.edu\", \n    \"cc\": \"vlarson@uwm.edu\", \n    \"resolution\": \"\", \n    \"_ts\": \"1526933965265788\", \n    \"component\": \"clubb_src\", \n    \"summary\": \"Modify ADG2 to control the sprawling standard deviation of chi in PDF component 1\", \n    \"priority\": \"major\", \n    \"keywords\": \"PDF chi\", \n    \"time\": \"2016-01-04T21:31:36\", \n    \"milestone\": \"Create and implement a new trivariate analytic double Gaussian PDF in CLUBB\", \n    \"owner\": \"bmg2@uwm.edu\", \n    \"type\": \"enhancement\"\n}\n```\n"},"comment":{"url":"https://api.github.com/repos/StrikeNP/trac_test/issues/comments/397814500","html_url":"https://github.com/StrikeNP/trac_test/issues/777#issuecomment-397814500","issue_url":"https://api.github.com/repos/StrikeNP/trac_test/issues/777","id":397814500,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NzgxNDUwMA==","user":{"login":"StrikeNP","id":2577059,"node_id":"MDQ6VXNlcjI1NzcwNTk=","avatar_url":"https://avatars0.githubusercontent.com/u/2577059?v=4","gravatar_id":"","url":"https://api.github.com/users/StrikeNP","html_url":"https://github.com/StrikeNP","followers_url":"https://api.github.com/users/StrikeNP/followers","following_url":"https://api.github.com/users/StrikeNP/following{/other_user}","gists_url":"https://api.github.com/users/StrikeNP/gists{/gist_id}","starred_url":"https://api.github.com/users/StrikeNP/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StrikeNP/subscriptions","organizations_url":"https://api.github.com/users/StrikeNP/orgs","repos_url":"https://api.github.com/users/StrikeNP/repos","events_url":"https://api.github.com/users/StrikeNP/events{/privacy}","received_events_url":"https://api.github.com/users/StrikeNP/received_events","type":"User","site_admin":false},"created_at":"2018-06-16T14:09:19Z","updated_at":"2018-06-16T14:09:19Z","author_association":"OWNER","body":"**Trac update on 2017/09/29 at 00:10**\n**bmg2@uwm.edu commented:**\n\nCommit comment from clubb:r8467:\n\nI added statistical output for F_w, F_rt, F_thl, min_F_w, max_F_w, min_F_rt,\nmax_F_rt, min_F_thl, and max_F_thl.\n\nThe results in GrADS output files from this revision are an exact match to the\nresults in GrADS output files from the previous revision for all cases for all\noutput files besides the *_zt.dat output files.  Owing to the new stats, the\n*_zt.dat output files make this revision BIT_CHANGING.\n\nCLUBB Trac ticket reference: https://github.com/larson-group/clubb/issues/777.\n\n\n\n"}}
{"actor":{"display_login":"StrikeNP"}}