{"action":"created","issue":{"url":"https://api.github.com/repos/nco/nco/issues/70","repository_url":"https://api.github.com/repos/nco/nco","labels_url":"https://api.github.com/repos/nco/nco/issues/70/labels{/name}","comments_url":"https://api.github.com/repos/nco/nco/issues/70/comments","events_url":"https://api.github.com/repos/nco/nco/issues/70/events","html_url":"https://github.com/nco/nco/issues/70","id":297942386,"number":70,"title":"Benchmark chunking","user":{"login":"czender","id":5597049,"avatar_url":"https://avatars0.githubusercontent.com/u/5597049?v=4","gravatar_id":"","url":"https://api.github.com/users/czender","html_url":"https://github.com/czender","followers_url":"https://api.github.com/users/czender/followers","following_url":"https://api.github.com/users/czender/following{/other_user}","gists_url":"https://api.github.com/users/czender/gists{/gist_id}","starred_url":"https://api.github.com/users/czender/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/czender/subscriptions","organizations_url":"https://api.github.com/users/czender/orgs","repos_url":"https://api.github.com/users/czender/repos","events_url":"https://api.github.com/users/czender/events{/privacy}","received_events_url":"https://api.github.com/users/czender/received_events","type":"User","site_admin":false},"labels":[{"id":824068594,"url":"https://api.github.com/repos/nco/nco/labels/medium%20priority","name":"medium priority","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":{"login":"hmb1","id":6428656,"avatar_url":"https://avatars0.githubusercontent.com/u/6428656?v=4","gravatar_id":"","url":"https://api.github.com/users/hmb1","html_url":"https://github.com/hmb1","followers_url":"https://api.github.com/users/hmb1/followers","following_url":"https://api.github.com/users/hmb1/following{/other_user}","gists_url":"https://api.github.com/users/hmb1/gists{/gist_id}","starred_url":"https://api.github.com/users/hmb1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hmb1/subscriptions","organizations_url":"https://api.github.com/users/hmb1/orgs","repos_url":"https://api.github.com/users/hmb1/repos","events_url":"https://api.github.com/users/hmb1/events{/privacy}","received_events_url":"https://api.github.com/users/hmb1/received_events","type":"User","site_admin":false},"assignees":[{"login":"hmb1","id":6428656,"avatar_url":"https://avatars0.githubusercontent.com/u/6428656?v=4","gravatar_id":"","url":"https://api.github.com/users/hmb1","html_url":"https://github.com/hmb1","followers_url":"https://api.github.com/users/hmb1/followers","following_url":"https://api.github.com/users/hmb1/following{/other_user}","gists_url":"https://api.github.com/users/hmb1/gists{/gist_id}","starred_url":"https://api.github.com/users/hmb1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hmb1/subscriptions","organizations_url":"https://api.github.com/users/hmb1/orgs","repos_url":"https://api.github.com/users/hmb1/repos","events_url":"https://api.github.com/users/hmb1/events{/privacy}","received_events_url":"https://api.github.com/users/hmb1/received_events","type":"User","site_admin":false}],"milestone":null,"comments":8,"created_at":"2018-02-16T22:41:03Z","updated_at":"2018-03-05T10:19:36Z","closed_at":null,"author_association":"OWNER","body":"Please design and implement an input data set\r\nexactly the same as described in\r\n\r\nhttps://www.unidata.ucar.edu/blogs/developer/en/entry/chunking_data_choosing_shapes\r\n\r\nand construct a script that runs ncks, ncccopy,\r\nand h5repack to (re-)chunk and the dataset\r\nexactly as in the tables in that page.\r\n\r\n1. Verify that the NCO rew algorithm\r\nproduces the same chunksizes as in Table 1.\r\n\r\n2. Then re-run tests in Table 2, where now\r\nthere's a third column for ncks with rew.\r\nnccopy and ncks should have similar timings.\r\n\r\n3. Then re-do the timings in Table 3 with\r\nncks and rew\r\n"},"comment":{"url":"https://api.github.com/repos/nco/nco/issues/comments/370373532","html_url":"https://github.com/nco/nco/issues/70#issuecomment-370373532","issue_url":"https://api.github.com/repos/nco/nco/issues/70","id":370373532,"user":{"login":"hmb1","id":6428656,"avatar_url":"https://avatars0.githubusercontent.com/u/6428656?v=4","gravatar_id":"","url":"https://api.github.com/users/hmb1","html_url":"https://github.com/hmb1","followers_url":"https://api.github.com/users/hmb1/followers","following_url":"https://api.github.com/users/hmb1/following{/other_user}","gists_url":"https://api.github.com/users/hmb1/gists{/gist_id}","starred_url":"https://api.github.com/users/hmb1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hmb1/subscriptions","organizations_url":"https://api.github.com/users/hmb1/orgs","repos_url":"https://api.github.com/users/hmb1/repos","events_url":"https://api.github.com/users/hmb1/events{/privacy}","received_events_url":"https://api.github.com/users/hmb1/received_events","type":"User","site_admin":false},"created_at":"2018-03-05T10:19:36Z","updated_at":"2018-03-05T10:19:36Z","author_association":"CONTRIBUTOR","body":"**why is nccopy faster than ncks for rechunking this large variable**\r\n\r\nI am with you on the variable level cache.\r\nI guess deep within the netcdf library is the function that  sequentially chunk data from the nc_get_vara(). \r\nif this function isn't needed then thats a speed up.  Also with ncks there's the system overhead of malloing 38GB of memory \r\n\r\n\r\n"}}
{"actor":{"display_login":"hmb1"}}