{"action":"created","issue":{"url":"https://api.github.com/repos/cf-convention/cf-conventions/issues/147","repository_url":"https://api.github.com/repos/cf-convention/cf-conventions","labels_url":"https://api.github.com/repos/cf-convention/cf-conventions/issues/147/labels{/name}","comments_url":"https://api.github.com/repos/cf-convention/cf-conventions/issues/147/comments","events_url":"https://api.github.com/repos/cf-convention/cf-conventions/issues/147/events","html_url":"https://github.com/cf-convention/cf-conventions/issues/147","id":368625141,"node_id":"MDU6SXNzdWUzNjg2MjUxNDE=","number":147,"title":"Clarify the use of compressed dimensions in related variables","user":{"login":"davidhassell","id":8126576,"node_id":"MDQ6VXNlcjgxMjY1NzY=","avatar_url":"https://avatars2.githubusercontent.com/u/8126576?v=4","gravatar_id":"","url":"https://api.github.com/users/davidhassell","html_url":"https://github.com/davidhassell","followers_url":"https://api.github.com/users/davidhassell/followers","following_url":"https://api.github.com/users/davidhassell/following{/other_user}","gists_url":"https://api.github.com/users/davidhassell/gists{/gist_id}","starred_url":"https://api.github.com/users/davidhassell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davidhassell/subscriptions","organizations_url":"https://api.github.com/users/davidhassell/orgs","repos_url":"https://api.github.com/users/davidhassell/repos","events_url":"https://api.github.com/users/davidhassell/events{/privacy}","received_events_url":"https://api.github.com/users/davidhassell/received_events","type":"User","site_admin":false},"labels":[{"id":595840613,"node_id":"MDU6TGFiZWw1OTU4NDA2MTM=","url":"https://api.github.com/repos/cf-convention/cf-conventions/labels/defect","name":"defect","color":"006b75","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-10-10T11:43:20Z","updated_at":"2018-10-11T12:49:16Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"### The problem\r\n\r\nWhen a data variable has dimensions that have been compressed by gathering (http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html#compression-by-gathering), the conventions currently make no statement on whether or not any related auxiliary coordinate, cell measure or ancillary variables that spans those dimensions must also be compressed, or whether such attached variables may be compressed across a different list of dimensions.\r\n\r\nThe situation is, however, clarified in the conformance document (http://cfconventions.org/Data/cf-documents/requirements-recommendations/requirements-recommendations-1.7.html), which states that attached variables must span a subset of the actual dimensions spanned by the data variable (sections 5 and 7.2). There are exceptions to this rule for auxiliary coordinate variables, for character arrays and DSG ragged arrays, but they do not apply to compression by gathering.\r\n\r\n### Proposed solution\r\n\r\nI propose that the conformance document is correct, and we would be fixing a defect in the conventions to clarify this in the main text.\r\n\r\n### Backwards compatibility\r\nI see no problems, because If there are existing datasets that apply compression differently to related variables than to the data variable, then these datasets would already fail the tests from the conformance document.\r\n\r\n### Changes required\r\n\r\n#### A. Add a third paragraph to section 8.2\r\n\r\n```\r\nAny auxillary coordinate, cell measure or ancillary variables related\r\nto a compressed data variable must not be compressed unless it can be\r\ncompressed using the same list variable indices as used by the data\r\nvariable, in which case compression must be applied. This will occur\r\nfor a related variable for which, when uncompressed, the individual\r\ncompressed axes appear adjacently and in the same order as the data\r\nvariable.\r\n```\r\n\r\n#### B. Update Example 8.1 to include a 2-d auxiliary coordinate variable:\r\n\r\nExample 8.1. Horizontal compression of a three-dimensional array\r\n\r\n```\r\ndimensions:\r\n  lat=73;\r\n  lon=96;\r\n  landpoint=2381;\r\n  depth=4;\r\nvariables:\r\n  int landpoint(landpoint);\r\n    landpoint:compress=\"lat lon\";\r\n  float landsoilt(depth,landpoint);\r\n    landsoilt:long_name=\"soil temperature\";\r\n    landsoilt:units=\"K\";\r\n    landsoilt:coordinates=\"soil_type\";\r\n  float depth(depth);\r\n  float lat(lat);\r\n  float lon(lon);\r\n  int soil_type(landpoint);\r\n     soil_type:long_name=\"integer code defining the soil type\";\r\ndata:\r\n  landpoint=363, 364, 365, ...;\r\n```\r\n  "},"comment":{"url":"https://api.github.com/repos/cf-convention/cf-conventions/issues/comments/428942389","html_url":"https://github.com/cf-convention/cf-conventions/issues/147#issuecomment-428942389","issue_url":"https://api.github.com/repos/cf-convention/cf-conventions/issues/147","id":428942389,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODk0MjM4OQ==","user":{"login":"davidhassell","id":8126576,"node_id":"MDQ6VXNlcjgxMjY1NzY=","avatar_url":"https://avatars2.githubusercontent.com/u/8126576?v=4","gravatar_id":"","url":"https://api.github.com/users/davidhassell","html_url":"https://github.com/davidhassell","followers_url":"https://api.github.com/users/davidhassell/followers","following_url":"https://api.github.com/users/davidhassell/following{/other_user}","gists_url":"https://api.github.com/users/davidhassell/gists{/gist_id}","starred_url":"https://api.github.com/users/davidhassell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davidhassell/subscriptions","organizations_url":"https://api.github.com/users/davidhassell/orgs","repos_url":"https://api.github.com/users/davidhassell/repos","events_url":"https://api.github.com/users/davidhassell/events{/privacy}","received_events_url":"https://api.github.com/users/davidhassell/received_events","type":"User","site_admin":false},"created_at":"2018-10-11T12:49:16Z","updated_at":"2018-10-11T12:49:16Z","author_association":"CONTRIBUTOR","body":"Sorry - this is more complicated than I first thought! The clarification needs to be redesigned, which I shall do shortly.\r\n\r\nThe issue I've realised is that we need to allow auxiliary coordinates and other related variables to span a subset of the compressed axes, but not be compressed themselves. In the following example, this would be the case for a 1-d auxiliary coordinate `aux_var` that spans the `lat` dimension:\r\n\r\n```\r\ndimensions:\r\n  lat=73;\r\n  lon=96;\r\n  landpoint=2381;\r\n  depth=4;\r\nvariables:\r\n  int landpoint(landpoint);\r\n    landpoint:compress=\"lat lon\";\r\n  float landsoilt(depth,landpoint);\r\n    landsoilt:long_name=\"soil temperature\";\r\n    landsoilt:units=\"K\";\r\n    landsoilt:coordinates=\"soil_type aux_var\";\r\n  float depth(depth);\r\n  float lat(lat);\r\n  float lon(lon);\r\n  int soil_type(landpoint);\r\n     soil_type:long_name=\"integer code defining the soil type\"; \r\n  float aux_var(lat);\r\n```\r\n\r\nIf this is allowed, then it would be the conformance document that is wrong, rather than the conventions document, as we are allowing the related variable to span a dimension (`lat`) that is not explicitly spanned by the data variable (which spans `depth` and `landpoint`)."}}
{"actor":{"display_login":"davidhassell"}}