{"action":"created","comment":{"url":"https://api.github.com/repos/SciTools/iris/pulls/comments/83395360","pull_request_review_id":4242610,"id":83395360,"diff_hunk":"@@ -1831,6 +1848,54 @@ def _create_cf_data_variable(self, cube, dimension_names, local_keys=None,\n             The newly created CF-netCDF data variable.\n \n         \"\"\"\n+        if 'packing' in kwargs:\n+            packing = kwargs.pop('packing')\n+            if packing:\n+                scale_factor = None\n+                add_offset = None\n+                fill_value = None\n+                masked = isinstance(cube.data, np.ma.core.MaskedArray)\n+                if isinstance(packing, dict):\n+                    if 'dtype' not in packing:\n+                        msg = (\"The dtype attribute is required for packing. \")","path":"lib/iris/fileformats/netcdf.py","position":55,"original_position":55,"commit_id":"e8d292a91670c4c51cc81cc02e320131c9c882a0","original_commit_id":"e8d292a91670c4c51cc81cc02e320131c9c882a0","user":{"login":"jswanljung","id":176077,"avatar_url":"https://avatars.githubusercontent.com/u/176077?v=3","gravatar_id":"","url":"https://api.github.com/users/jswanljung","html_url":"https://github.com/jswanljung","followers_url":"https://api.github.com/users/jswanljung/followers","following_url":"https://api.github.com/users/jswanljung/following{/other_user}","gists_url":"https://api.github.com/users/jswanljung/gists{/gist_id}","starred_url":"https://api.github.com/users/jswanljung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jswanljung/subscriptions","organizations_url":"https://api.github.com/users/jswanljung/orgs","repos_url":"https://api.github.com/users/jswanljung/repos","events_url":"https://api.github.com/users/jswanljung/events{/privacy}","received_events_url":"https://api.github.com/users/jswanljung/received_events","type":"User","site_admin":false},"body":"Whoops. I think it was longer before...","created_at":"2016-10-14T10:20:03Z","updated_at":"2016-10-14T10:20:03Z","html_url":"https://github.com/SciTools/iris/pull/2193#discussion_r83395360","pull_request_url":"https://api.github.com/repos/SciTools/iris/pulls/2193","_links":{"self":{"href":"https://api.github.com/repos/SciTools/iris/pulls/comments/83395360"},"html":{"href":"https://github.com/SciTools/iris/pull/2193#discussion_r83395360"},"pull_request":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2193"}}},"pull_request":{"url":"https://api.github.com/repos/SciTools/iris/pulls/2193","id":89207747,"html_url":"https://github.com/SciTools/iris/pull/2193","diff_url":"https://github.com/SciTools/iris/pull/2193.diff","patch_url":"https://github.com/SciTools/iris/pull/2193.patch","issue_url":"https://api.github.com/repos/SciTools/iris/issues/2193","number":2193,"state":"open","locked":false,"title":"Netcdf packing support v2","user":{"login":"jswanljung","id":176077,"avatar_url":"https://avatars.githubusercontent.com/u/176077?v=3","gravatar_id":"","url":"https://api.github.com/users/jswanljung","html_url":"https://github.com/jswanljung","followers_url":"https://api.github.com/users/jswanljung/followers","following_url":"https://api.github.com/users/jswanljung/following{/other_user}","gists_url":"https://api.github.com/users/jswanljung/gists{/gist_id}","starred_url":"https://api.github.com/users/jswanljung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jswanljung/subscriptions","organizations_url":"https://api.github.com/users/jswanljung/orgs","repos_url":"https://api.github.com/users/jswanljung/repos","events_url":"https://api.github.com/users/jswanljung/events{/privacy}","received_events_url":"https://api.github.com/users/jswanljung/received_events","type":"User","site_admin":false},"body":"NetCDF has support for variable packing; by specifying attributes scale_factor and add_offset, data can be packed into smaller data types. 16 bit short integers provide more than enough quantization for suitably scaled temperature data, for instance. See:\r\nhttp://unidata.ucar.edu/software/netcdf/docs/BestPractices.html#bp_Packed-Data-Values \r\nIris already supports loading of packed netCDF because it is automatically handled by the netCDF4 module. However, there is currently no way to save data with packing.\r\n\r\nThis PR implements packing by adding an optional `packing` argument to the netcdf.save function and netcdf.Saver.write method. As described in the docstring:\r\n \r\n- packing (type or string or dict or list): A numpy integer datatype\r\n    (signed or unsigned) or a string that describes a numpy integer dtype\r\n    (i.e. 'i2', 'short', 'u4') or a dict of packing parameters as described\r\n    below or a list of such types, strings, or dicts. This provides support\r\n    for netCDF data packing as described in\r\n    http://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html#bp_Packed-Data-Values\r\n    If this argument is a type (or type string), appropriate values of\r\n    scale_factor and add_offset will be automatically calculated based on\r\n    `cube.data` and possible masking. For masked data, `fill_value` is\r\n    taken from netCDF4.default_fillvals. For more control, pass a dict with\r\n    one or more of the following keys: `dtype` (required), `scale_factor`,\r\n    `add_offset`, and `fill_value`. To save multiple cubes with different\r\n    packing parameters, pass a list of types, strings, dicts, or `None`,\r\n    one for each cube.  Note that automatic calculation of packing\r\n    parameters will trigger loading of lazy data; set them manually using a\r\n    dict to avoid this. The default is `None`, in which case the datatype\r\n    is determined from the cube and no packing will occur.\r\n\r\nSo for example:\r\n```\r\n# pack into np.int16 with automatic calc of scale_factor and add_offset, default fillval if applicable\r\n iris.save(cube, 'outfile.nc', packing='int16') \r\n\r\n# pack into np.int16 with manually set scale_factor and add_offset\r\niris.save(cube, 'outfile.nc', packing=dict(dtype=np.short, scale_factor=0.01,  add_offset=250))\r\n\r\n# pack one cube with auto scale, the other manually, and the third not at all (cubelist has 3 cubes)\r\niris.save(cubelist, 'outfile.nc', packing=['int16', dict(dtype='int16', scale_factor=0.01), None])\r\n```\r\n\r\nThis PR supercedes a previous attempt (https://github.com/SciTools/iris/pull/2152) which I withdrew because it set packing attributes in the cube. In this version packing is completely decoupled from\r\nthe cube (see the previous PR for discussion). It also uses the new pattern for  setting cf attributes (change requested by @marqh).\r\n\r\nIn the discussion in the previous PR, @ajdawson objected (and I agreed) that the method implemented here for providing different packing specifications for multiple cubes is suboptimal. Ideally we would identify cubes in the list by key, but we were unable to come up with keys that could identify cubes uniquely in a list. There is no requirement that cube names be unique, for instance. Suggestions are welcome.","created_at":"2016-10-13T13:59:08Z","updated_at":"2016-10-14T10:20:03Z","closed_at":null,"merged_at":null,"merge_commit_sha":"80ef7031b86343c3e7b9751f83c982dd13ff613c","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/SciTools/iris/pulls/2193/commits","review_comments_url":"https://api.github.com/repos/SciTools/iris/pulls/2193/comments","review_comment_url":"https://api.github.com/repos/SciTools/iris/pulls/comments{/number}","comments_url":"https://api.github.com/repos/SciTools/iris/issues/2193/comments","statuses_url":"https://api.github.com/repos/SciTools/iris/statuses/e8d292a91670c4c51cc81cc02e320131c9c882a0","head":{"label":"jswanljung:netcdf_packing_support_v2","ref":"netcdf_packing_support_v2","sha":"e8d292a91670c4c51cc81cc02e320131c9c882a0","user":{"login":"jswanljung","id":176077,"avatar_url":"https://avatars.githubusercontent.com/u/176077?v=3","gravatar_id":"","url":"https://api.github.com/users/jswanljung","html_url":"https://github.com/jswanljung","followers_url":"https://api.github.com/users/jswanljung/followers","following_url":"https://api.github.com/users/jswanljung/following{/other_user}","gists_url":"https://api.github.com/users/jswanljung/gists{/gist_id}","starred_url":"https://api.github.com/users/jswanljung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jswanljung/subscriptions","organizations_url":"https://api.github.com/users/jswanljung/orgs","repos_url":"https://api.github.com/users/jswanljung/repos","events_url":"https://api.github.com/users/jswanljung/events{/privacy}","received_events_url":"https://api.github.com/users/jswanljung/received_events","type":"User","site_admin":false},"repo":{"id":68528573,"name":"iris","full_name":"jswanljung/iris","owner":{"login":"jswanljung","id":176077,"avatar_url":"https://avatars.githubusercontent.com/u/176077?v=3","gravatar_id":"","url":"https://api.github.com/users/jswanljung","html_url":"https://github.com/jswanljung","followers_url":"https://api.github.com/users/jswanljung/followers","following_url":"https://api.github.com/users/jswanljung/following{/other_user}","gists_url":"https://api.github.com/users/jswanljung/gists{/gist_id}","starred_url":"https://api.github.com/users/jswanljung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jswanljung/subscriptions","organizations_url":"https://api.github.com/users/jswanljung/orgs","repos_url":"https://api.github.com/users/jswanljung/repos","events_url":"https://api.github.com/users/jswanljung/events{/privacy}","received_events_url":"https://api.github.com/users/jswanljung/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jswanljung/iris","description":"A powerful, easy to use, and community-driven Python library for analysing and visualising meteorological and oceanographic data sets.","fork":true,"url":"https://api.github.com/repos/jswanljung/iris","forks_url":"https://api.github.com/repos/jswanljung/iris/forks","keys_url":"https://api.github.com/repos/jswanljung/iris/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jswanljung/iris/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jswanljung/iris/teams","hooks_url":"https://api.github.com/repos/jswanljung/iris/hooks","issue_events_url":"https://api.github.com/repos/jswanljung/iris/issues/events{/number}","events_url":"https://api.github.com/repos/jswanljung/iris/events","assignees_url":"https://api.github.com/repos/jswanljung/iris/assignees{/user}","branches_url":"https://api.github.com/repos/jswanljung/iris/branches{/branch}","tags_url":"https://api.github.com/repos/jswanljung/iris/tags","blobs_url":"https://api.github.com/repos/jswanljung/iris/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jswanljung/iris/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jswanljung/iris/git/refs{/sha}","trees_url":"https://api.github.com/repos/jswanljung/iris/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jswanljung/iris/statuses/{sha}","languages_url":"https://api.github.com/repos/jswanljung/iris/languages","stargazers_url":"https://api.github.com/repos/jswanljung/iris/stargazers","contributors_url":"https://api.github.com/repos/jswanljung/iris/contributors","subscribers_url":"https://api.github.com/repos/jswanljung/iris/subscribers","subscription_url":"https://api.github.com/repos/jswanljung/iris/subscription","commits_url":"https://api.github.com/repos/jswanljung/iris/commits{/sha}","git_commits_url":"https://api.github.com/repos/jswanljung/iris/git/commits{/sha}","comments_url":"https://api.github.com/repos/jswanljung/iris/comments{/number}","issue_comment_url":"https://api.github.com/repos/jswanljung/iris/issues/comments{/number}","contents_url":"https://api.github.com/repos/jswanljung/iris/contents/{+path}","compare_url":"https://api.github.com/repos/jswanljung/iris/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jswanljung/iris/merges","archive_url":"https://api.github.com/repos/jswanljung/iris/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jswanljung/iris/downloads","issues_url":"https://api.github.com/repos/jswanljung/iris/issues{/number}","pulls_url":"https://api.github.com/repos/jswanljung/iris/pulls{/number}","milestones_url":"https://api.github.com/repos/jswanljung/iris/milestones{/number}","notifications_url":"https://api.github.com/repos/jswanljung/iris/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jswanljung/iris/labels{/name}","releases_url":"https://api.github.com/repos/jswanljung/iris/releases{/id}","deployments_url":"https://api.github.com/repos/jswanljung/iris/deployments","created_at":"2016-09-18T15:04:42Z","updated_at":"2016-09-18T15:04:47Z","pushed_at":"2016-10-13T13:35:49Z","git_url":"git://github.com/jswanljung/iris.git","ssh_url":"git@github.com:jswanljung/iris.git","clone_url":"https://github.com/jswanljung/iris.git","svn_url":"https://github.com/jswanljung/iris","homepage":"http://scitools.org.uk/iris/","size":101887,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"netcdf_packing_support"}},"base":{"label":"SciTools:master","ref":"master","sha":"5645de0b443f988c7a029e3053ef2a898d87f74c","user":{"login":"SciTools","id":1391487,"avatar_url":"https://avatars.githubusercontent.com/u/1391487?v=3","gravatar_id":"","url":"https://api.github.com/users/SciTools","html_url":"https://github.com/SciTools","followers_url":"https://api.github.com/users/SciTools/followers","following_url":"https://api.github.com/users/SciTools/following{/other_user}","gists_url":"https://api.github.com/users/SciTools/gists{/gist_id}","starred_url":"https://api.github.com/users/SciTools/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SciTools/subscriptions","organizations_url":"https://api.github.com/users/SciTools/orgs","repos_url":"https://api.github.com/users/SciTools/repos","events_url":"https://api.github.com/users/SciTools/events{/privacy}","received_events_url":"https://api.github.com/users/SciTools/received_events","type":"Organization","site_admin":false},"repo":{"id":5312648,"name":"iris","full_name":"SciTools/iris","owner":{"login":"SciTools","id":1391487,"avatar_url":"https://avatars.githubusercontent.com/u/1391487?v=3","gravatar_id":"","url":"https://api.github.com/users/SciTools","html_url":"https://github.com/SciTools","followers_url":"https://api.github.com/users/SciTools/followers","following_url":"https://api.github.com/users/SciTools/following{/other_user}","gists_url":"https://api.github.com/users/SciTools/gists{/gist_id}","starred_url":"https://api.github.com/users/SciTools/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SciTools/subscriptions","organizations_url":"https://api.github.com/users/SciTools/orgs","repos_url":"https://api.github.com/users/SciTools/repos","events_url":"https://api.github.com/users/SciTools/events{/privacy}","received_events_url":"https://api.github.com/users/SciTools/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/SciTools/iris","description":"A powerful, easy to use, and community-driven Python library for analysing and visualising meteorological and oceanographic data sets.","fork":false,"url":"https://api.github.com/repos/SciTools/iris","forks_url":"https://api.github.com/repos/SciTools/iris/forks","keys_url":"https://api.github.com/repos/SciTools/iris/keys{/key_id}","collaborators_url":"https://api.github.com/repos/SciTools/iris/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/SciTools/iris/teams","hooks_url":"https://api.github.com/repos/SciTools/iris/hooks","issue_events_url":"https://api.github.com/repos/SciTools/iris/issues/events{/number}","events_url":"https://api.github.com/repos/SciTools/iris/events","assignees_url":"https://api.github.com/repos/SciTools/iris/assignees{/user}","branches_url":"https://api.github.com/repos/SciTools/iris/branches{/branch}","tags_url":"https://api.github.com/repos/SciTools/iris/tags","blobs_url":"https://api.github.com/repos/SciTools/iris/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/SciTools/iris/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/SciTools/iris/git/refs{/sha}","trees_url":"https://api.github.com/repos/SciTools/iris/git/trees{/sha}","statuses_url":"https://api.github.com/repos/SciTools/iris/statuses/{sha}","languages_url":"https://api.github.com/repos/SciTools/iris/languages","stargazers_url":"https://api.github.com/repos/SciTools/iris/stargazers","contributors_url":"https://api.github.com/repos/SciTools/iris/contributors","subscribers_url":"https://api.github.com/repos/SciTools/iris/subscribers","subscription_url":"https://api.github.com/repos/SciTools/iris/subscription","commits_url":"https://api.github.com/repos/SciTools/iris/commits{/sha}","git_commits_url":"https://api.github.com/repos/SciTools/iris/git/commits{/sha}","comments_url":"https://api.github.com/repos/SciTools/iris/comments{/number}","issue_comment_url":"https://api.github.com/repos/SciTools/iris/issues/comments{/number}","contents_url":"https://api.github.com/repos/SciTools/iris/contents/{+path}","compare_url":"https://api.github.com/repos/SciTools/iris/compare/{base}...{head}","merges_url":"https://api.github.com/repos/SciTools/iris/merges","archive_url":"https://api.github.com/repos/SciTools/iris/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/SciTools/iris/downloads","issues_url":"https://api.github.com/repos/SciTools/iris/issues{/number}","pulls_url":"https://api.github.com/repos/SciTools/iris/pulls{/number}","milestones_url":"https://api.github.com/repos/SciTools/iris/milestones{/number}","notifications_url":"https://api.github.com/repos/SciTools/iris/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/SciTools/iris/labels{/name}","releases_url":"https://api.github.com/repos/SciTools/iris/releases{/id}","deployments_url":"https://api.github.com/repos/SciTools/iris/deployments","created_at":"2012-08-06T10:09:14Z","updated_at":"2016-10-10T16:52:17Z","pushed_at":"2016-10-14T10:05:30Z","git_url":"git://github.com/SciTools/iris.git","ssh_url":"git@github.com:SciTools/iris.git","clone_url":"https://github.com/SciTools/iris.git","svn_url":"https://github.com/SciTools/iris","homepage":"http://scitools.org.uk/iris/","size":145382,"stargazers_count":140,"watchers_count":140,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":103,"mirror_url":null,"open_issues_count":248,"forks":103,"open_issues":248,"watchers":140,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2193"},"html":{"href":"https://github.com/SciTools/iris/pull/2193"},"issue":{"href":"https://api.github.com/repos/SciTools/iris/issues/2193"},"comments":{"href":"https://api.github.com/repos/SciTools/iris/issues/2193/comments"},"review_comments":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2193/comments"},"review_comment":{"href":"https://api.github.com/repos/SciTools/iris/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/SciTools/iris/pulls/2193/commits"},"statuses":{"href":"https://api.github.com/repos/SciTools/iris/statuses/e8d292a91670c4c51cc81cc02e320131c9c882a0"}}}}
{"actor":{"display_login":"jswanljung"}}