{"action":"created","issue":{"url":"https://api.github.com/repos/xgcm/xgcm/issues/40","repository_url":"https://api.github.com/repos/xgcm/xgcm","labels_url":"https://api.github.com/repos/xgcm/xgcm/issues/40/labels{/name}","comments_url":"https://api.github.com/repos/xgcm/xgcm/issues/40/comments","events_url":"https://api.github.com/repos/xgcm/xgcm/issues/40/events","html_url":"https://github.com/xgcm/xgcm/issues/40","id":160546787,"number":40,"title":"Cannot allocate memory error","user":{"login":"jklymak","id":1562854,"avatar_url":"https://avatars.githubusercontent.com/u/1562854?v=3","gravatar_id":"","url":"https://api.github.com/users/jklymak","html_url":"https://github.com/jklymak","followers_url":"https://api.github.com/users/jklymak/followers","following_url":"https://api.github.com/users/jklymak/following{/other_user}","gists_url":"https://api.github.com/users/jklymak/gists{/gist_id}","starred_url":"https://api.github.com/users/jklymak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jklymak/subscriptions","organizations_url":"https://api.github.com/users/jklymak/orgs","repos_url":"https://api.github.com/users/jklymak/repos","events_url":"https://api.github.com/users/jklymak/events{/privacy}","received_events_url":"https://api.github.com/users/jklymak/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-06-15T23:30:13Z","updated_at":"2016-10-16T13:40:36Z","closed_at":"2016-10-16T13:40:36Z","body":"Hi Ryan,  Thanks for your help so far.  Another quick question, largely so I understand what is going on:\r\n\r\nRunning \r\n```python\r\nimport numpy as np\r\nimport xarray\r\nimport dask.array as daskarray\r\nimport xgcm\r\n\r\nddir = '/scr/jklymak/LeeWave3d/lee3dscpuio/_Model/input'\r\niters = [720,]\r\nds = xgcm.open_mdsdataset(ddir, iters, deltaT=5.,prefix=['U','V','W','T','PH'],endian='<').chunk()\r\n\r\nprint ds\r\n```\r\n I get a memory error on the head node of my cluster.  On a compute node (i.e. if I submit the job) it works fine, so this is more for my curiosity.  If I haven't addressed any of the data yet, why does the memory allocation become a problem?\r\n```\r\npython testxgcm.py\r\n/u/home/jklymak/anaconda2/lib/python2.7/site-packages/xgcm/mdsxray.py:273: UserWarning: Not sure what to do with rlev = L\r\n  warnings.warn(\"Not sure what to do with rlev = \" + rlev)\r\nTraceback (most recent call last):\r\n  File \"testxgcm.py\", line 10, in <module>\r\n    ds = xgcm.open_mdsdataset(ddir, iters, deltaT=5.,prefix=['U','V','W','T','PH'],endian='<').chunk()\r\n  File \"/u/home/jklymak/anaconda2/lib/python2.7/site-packages/xgcm/mdsxray.py\", line 487, in open_mdsdataset\r\n    ignore_pickup, geometry, skip_vars, endian)\r\n  File \"/u/home/jklymak/anaconda2/lib/python2.7/site-packages/xgcm/mdsxray.py\", line 610, in __init__\r\n    data = _read_mds(f, i, force_dict=True, endian=endian)\r\n  File \"/u/home/jklymak/anaconda2/lib/python2.7/site-packages/xgcm/mdsxray.py\", line 354, in _read_mds\r\n    d = np.memmap(datafile, dtype, 'r')\r\n  File \"/u/home/jklymak/anaconda2/lib/python2.7/site-packages/numpy/core/memmap.py\", line 260, in __new__\r\n    mm = mmap.mmap(fid.fileno(), bytes, access=acc, offset=start)\r\nmmap.error: [Errno 12] Cannot allocate memory\r\n```"},"comment":{"url":"https://api.github.com/repos/xgcm/xgcm/issues/comments/254047575","html_url":"https://github.com/xgcm/xgcm/issues/40#issuecomment-254047575","issue_url":"https://api.github.com/repos/xgcm/xgcm/issues/40","id":254047575,"user":{"login":"rabernat","id":1197350,"avatar_url":"https://avatars.githubusercontent.com/u/1197350?v=3","gravatar_id":"","url":"https://api.github.com/users/rabernat","html_url":"https://github.com/rabernat","followers_url":"https://api.github.com/users/rabernat/followers","following_url":"https://api.github.com/users/rabernat/following{/other_user}","gists_url":"https://api.github.com/users/rabernat/gists{/gist_id}","starred_url":"https://api.github.com/users/rabernat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rabernat/subscriptions","organizations_url":"https://api.github.com/users/rabernat/orgs","repos_url":"https://api.github.com/users/rabernat/repos","events_url":"https://api.github.com/users/rabernat/events{/privacy}","received_events_url":"https://api.github.com/users/rabernat/received_events","type":"User","site_admin":false},"created_at":"2016-10-16T13:40:36Z","updated_at":"2016-10-16T13:40:36Z","body":"@jklymak: I just wanted to give you a heads up that I have moved all of the mitgcm-specific features of xgcm into a new package called xgcm:\r\nhttps://github.com/xgcm/xmitgcm\r\nThis move also includes a major refactor of the code. I am closing the mitgcm-specific issues here, but I would really appreciate your input on the new package.\r\n\r\nxgcm is going back to it's original mission: implementing finite-volume differential and integral operations on GCM grids (which is not specific to mitgcm).\r\n\r\n"}}
{"actor":{"display_login":"rabernat"}}