{"action":"created","issue":{"url":"https://api.github.com/repos/ESMCI/cime/issues/396","repository_url":"https://api.github.com/repos/ESMCI/cime","labels_url":"https://api.github.com/repos/ESMCI/cime/issues/396/labels{/name}","comments_url":"https://api.github.com/repos/ESMCI/cime/issues/396/comments","events_url":"https://api.github.com/repos/ESMCI/cime/issues/396/events","html_url":"https://github.com/ESMCI/cime/issues/396","id":171037362,"number":396,"title":"TestStatus: Still would like clearer distinction between test comparisons and baseline comparisons","user":{"login":"billsacks","id":6266741,"avatar_url":"https://avatars.githubusercontent.com/u/6266741?v=3","gravatar_id":"","url":"https://api.github.com/users/billsacks","html_url":"https://github.com/billsacks","followers_url":"https://api.github.com/users/billsacks/followers","following_url":"https://api.github.com/users/billsacks/following{/other_user}","gists_url":"https://api.github.com/users/billsacks/gists{/gist_id}","starred_url":"https://api.github.com/users/billsacks/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/billsacks/subscriptions","organizations_url":"https://api.github.com/users/billsacks/orgs","repos_url":"https://api.github.com/users/billsacks/repos","events_url":"https://api.github.com/users/billsacks/events{/privacy}","received_events_url":"https://api.github.com/users/billsacks/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ESMCI/cime/labels/ready","name":"ready","color":"ededed"}],"state":"closed","locked":false,"assignee":{"login":"jgfouca","id":7292036,"avatar_url":"https://avatars.githubusercontent.com/u/7292036?v=3","gravatar_id":"","url":"https://api.github.com/users/jgfouca","html_url":"https://github.com/jgfouca","followers_url":"https://api.github.com/users/jgfouca/followers","following_url":"https://api.github.com/users/jgfouca/following{/other_user}","gists_url":"https://api.github.com/users/jgfouca/gists{/gist_id}","starred_url":"https://api.github.com/users/jgfouca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jgfouca/subscriptions","organizations_url":"https://api.github.com/users/jgfouca/orgs","repos_url":"https://api.github.com/users/jgfouca/repos","events_url":"https://api.github.com/users/jgfouca/events{/privacy}","received_events_url":"https://api.github.com/users/jgfouca/received_events","type":"User","site_admin":false},"assignees":[{"login":"jgfouca","id":7292036,"avatar_url":"https://avatars.githubusercontent.com/u/7292036?v=3","gravatar_id":"","url":"https://api.github.com/users/jgfouca","html_url":"https://github.com/jgfouca","followers_url":"https://api.github.com/users/jgfouca/followers","following_url":"https://api.github.com/users/jgfouca/following{/other_user}","gists_url":"https://api.github.com/users/jgfouca/gists{/gist_id}","starred_url":"https://api.github.com/users/jgfouca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jgfouca/subscriptions","organizations_url":"https://api.github.com/users/jgfouca/orgs","repos_url":"https://api.github.com/users/jgfouca/repos","events_url":"https://api.github.com/users/jgfouca/events{/privacy}","received_events_url":"https://api.github.com/users/jgfouca/received_events","type":"User","site_admin":false}],"milestone":null,"comments":1,"created_at":"2016-08-14T01:50:31Z","updated_at":"2016-10-06T20:47:54Z","closed_at":"2016-10-06T20:47:54Z","body":"Inspired by @gold2718 's comment here:\r\n\r\nhttps://github.com/ESMCI/cime/pull/392#issuecomment-239626725\r\n\r\n- where he suggests knowing ahead of time which test phases are expected: I realized that it would still be good to have a clearer distinction made between history file comparisons done as an integral part of a test (e.g., comparing .base with .rest files for an ERS test) vs. baseline comparisons. Even though these are both about comparing history files, they are fundamentally different in their meaning. \r\n\r\ncime4 test reporting made a clear distinction between these, as:\r\n\r\n```\r\n--- Test Functionality  ---:\r\nPASS ERS_D_Ld10.T31_g37_gl5.IGHISTCLM45.yellowstone_intel.clm-glcMEC_decrease.clm2.h0.nc : test compare clm2.h0 (.base and .rest files)\r\n<snip>\r\n--- Baseline Comparison ---:\r\nPASS ERS_D_Ld10.T31_g37_gl5.IGHISTCLM45.yellowstone_intel.clm-glcMEC_decrease.cism.h.nc : baseline compare cism.h (baseline: compare .base file with clm4_5_9_r185 file)\r\n```\r\n\r\nand I feel it would be good to maintain that clear distinction in cime5.\r\n\r\n@jgfouca made some steps towards this, by adding suffixes to the single 'COMPARE' phase. But it seems like it would be good to explicitly split this into two phases - a TEST_COMPARE phase and a BASELINE_COMPARE phase - both in the code and in the test reporting. In the test reporting, this will make it easier to do greps to include / exclude particular aspects of the tests. In the code, this will make it easier to add new functionality, such as the functionality proposed by @gold2718 ."},"comment":{"url":"https://api.github.com/repos/ESMCI/cime/issues/comments/252083908","html_url":"https://github.com/ESMCI/cime/issues/396#issuecomment-252083908","issue_url":"https://api.github.com/repos/ESMCI/cime/issues/396","id":252083908,"user":{"login":"jgfouca","id":7292036,"avatar_url":"https://avatars.githubusercontent.com/u/7292036?v=3","gravatar_id":"","url":"https://api.github.com/users/jgfouca","html_url":"https://github.com/jgfouca","followers_url":"https://api.github.com/users/jgfouca/followers","following_url":"https://api.github.com/users/jgfouca/following{/other_user}","gists_url":"https://api.github.com/users/jgfouca/gists{/gist_id}","starred_url":"https://api.github.com/users/jgfouca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jgfouca/subscriptions","organizations_url":"https://api.github.com/users/jgfouca/orgs","repos_url":"https://api.github.com/users/jgfouca/repos","events_url":"https://api.github.com/users/jgfouca/events{/privacy}","received_events_url":"https://api.github.com/users/jgfouca/received_events","type":"User","site_admin":false},"created_at":"2016-10-06T20:47:54Z","updated_at":"2016-10-06T20:47:54Z","body":"@billsacks the handling of COMPARE phases in CIME is already awkward and having another phase like COMPARE is not something I'm excited about. If you can describe a specific need/capability that would require this change, would you add a comment hear describing it? I'm going to tentatively close this for now. Feel free to reopen if you think we should reconsider."}}
{"actor":{"display_login":"jgfouca"}}