{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/xarray/issues/2223","repository_url":"https://api.github.com/repos/pydata/xarray","labels_url":"https://api.github.com/repos/pydata/xarray/issues/2223/labels{/name}","comments_url":"https://api.github.com/repos/pydata/xarray/issues/2223/comments","events_url":"https://api.github.com/repos/pydata/xarray/issues/2223/events","html_url":"https://github.com/pydata/xarray/issues/2223","id":330918967,"node_id":"MDU6SXNzdWUzMzA5MTg5Njc=","number":2223,"title":"DataArray.interp() : poor performance","user":{"login":"gwin-zegal","id":15956441,"node_id":"MDQ6VXNlcjE1OTU2NDQx","avatar_url":"https://avatars2.githubusercontent.com/u/15956441?v=4","gravatar_id":"","url":"https://api.github.com/users/gwin-zegal","html_url":"https://github.com/gwin-zegal","followers_url":"https://api.github.com/users/gwin-zegal/followers","following_url":"https://api.github.com/users/gwin-zegal/following{/other_user}","gists_url":"https://api.github.com/users/gwin-zegal/gists{/gist_id}","starred_url":"https://api.github.com/users/gwin-zegal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gwin-zegal/subscriptions","organizations_url":"https://api.github.com/users/gwin-zegal/orgs","repos_url":"https://api.github.com/users/gwin-zegal/repos","events_url":"https://api.github.com/users/gwin-zegal/events{/privacy}","received_events_url":"https://api.github.com/users/gwin-zegal/received_events","type":"User","site_admin":false},"labels":[{"id":536144505,"node_id":"MDU6TGFiZWw1MzYxNDQ1MDU=","url":"https://api.github.com/repos/pydata/xarray/labels/performance","name":"performance","color":"d93f0b","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-06-09T21:01:30Z","updated_at":"2018-06-10T13:42:59Z","closed_at":null,"author_association":"NONE","body":"#### Code Sample\r\n\r\nHello,\r\n\r\nI performed a quick comparison of the newly introduced method ìnterp()` with an adapter (draft) to the sdf (scientific data format) library: https://gist.github.com/gwin-zegal/b955c3ef63f5ad51eec6329dd2e620be#file-array_sdf_interp-py\r\n\r\nCode for a micro comparison (2D array) in python (include the above gist first):\r\n\r\n```python\r\n\r\narr = xr.DataArray(np.sort(np.sort(np.random.RandomState(123).rand(30,4), axis=0), axis=1),\r\n                   coords=[('tension', np.arange(10, 40)),\r\n                           ('resistance', np.linspace(100, 500, 4))])\r\n\r\nres = {'xarray': [], 'c_sdf' : []}\r\nx = np.logspace(1, 4, num=10, dtype=np.int16)\r\nfor size in x:\r\n    new_tension = arr.tension[0].data + np.random.random_sample(size=size) * (arr.tension[-1].data - arr.tension[0].data)\r\n    new_resistance = arr.resistance[0].data + np.random.random_sample(size=size) * (arr.resistance[-1].data - arr.resistance[0].data)\r\n\r\n    interp_xr = %timeit -qo arr.interp({'tension': new_tension, 'resistance': new_resistance})\r\n    res['xarray'].append(interp_xr)\r\n    \r\n    interp_c_sdf = %timeit -qo arr(new_tension, new_resistance)\r\n    res['c_sdf'].append(interp_c_sdf)\r\n```\r\n#### Problem description\r\n\r\nThe time spent for `array.interp()`is growing exponentially... over two 2min (xarray internal interp) on my old machine compared to 9ms (C-SDF wrapper) for 10_000 interpolations. \r\n\r\nThe C-SDF code is slow (a copy of the array is performed and algorithms not so optimized), but xarray implementation is not usable in daily life on my machine! \r\n\r\n{'xarray': [<TimeitResult : 6.46 ms ± 223 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 6.46 ms ± 88.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 6.99 ms ± 141 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 8.91 ms ± 52 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 19.8 ms ± 183 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 112 ms ± 638 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)>,\r\n  <TimeitResult : 584 ms ± 19.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)>,\r\n  <TimeitResult : 2.63 s ± 43.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)>,\r\n  <TimeitResult : 15.5 s ± 147 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)>,\r\n  <TimeitResult : 2min 23s ± 18.7 s per loop (mean ± std. dev. of 7 runs, 1 loop each)>],\r\n 'c_sdf': [<TimeitResult : 1.08 ms ± 7.96 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 1.09 ms ± 7.91 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 1.1 ms ± 12.9 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 1.13 ms ± 9.95 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 1.19 ms ± 7.76 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 1.32 ms ± 9.47 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 1.59 ms ± 10.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)>,\r\n  <TimeitResult : 2.19 ms ± 31.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 3.51 ms ± 34.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>,\r\n  <TimeitResult : 9.27 ms ± 307 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)>]}\r\n\r\nPerformance issue on my machine or is it confirmed by others?\r\n\r\n#### Output of ``xr.show_versions()``\r\n\r\n<details>\r\n\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 3.6.5.final.0\r\npython-bits: 64\r\nOS: Darwin\r\nOS-release: 15.6.0\r\nmachine: x86_64\r\nprocessor: i386\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: fr_FR.UTF-8\r\nLOCALE: fr_FR.UTF-8\r\n\r\nxarray: 0.10.7\r\npandas: 0.23.0\r\nnumpy: 1.14.3\r\nscipy: 1.1.0\r\nnetCDF4: None\r\nh5netcdf: None\r\nh5py: 2.8.0\r\nNio: None\r\nzarr: None\r\nbottleneck: 1.2.1\r\ncyordereddict: None\r\ndask: 0.17.5\r\ndistributed: 1.21.8\r\nmatplotlib: 2.2.2\r\ncartopy: 0.16.0\r\nseaborn: 0.8.1\r\nsetuptools: 39.2.0\r\npip: 10.0.1\r\nconda: 4.5.4\r\npytest: 3.4.1\r\nIPython: 6.4.0\r\nsphinx: 1.7.5\r\n\r\n</details>\r\n"},"comment":{"url":"https://api.github.com/repos/pydata/xarray/issues/comments/396050056","html_url":"https://github.com/pydata/xarray/issues/2223#issuecomment-396050056","issue_url":"https://api.github.com/repos/pydata/xarray/issues/2223","id":396050056,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NjA1MDA1Ng==","user":{"login":"fujiisoup","id":6815844,"node_id":"MDQ6VXNlcjY4MTU4NDQ=","avatar_url":"https://avatars2.githubusercontent.com/u/6815844?v=4","gravatar_id":"","url":"https://api.github.com/users/fujiisoup","html_url":"https://github.com/fujiisoup","followers_url":"https://api.github.com/users/fujiisoup/followers","following_url":"https://api.github.com/users/fujiisoup/following{/other_user}","gists_url":"https://api.github.com/users/fujiisoup/gists{/gist_id}","starred_url":"https://api.github.com/users/fujiisoup/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fujiisoup/subscriptions","organizations_url":"https://api.github.com/users/fujiisoup/orgs","repos_url":"https://api.github.com/users/fujiisoup/repos","events_url":"https://api.github.com/users/fujiisoup/events{/privacy}","received_events_url":"https://api.github.com/users/fujiisoup/received_events","type":"User","site_admin":false},"created_at":"2018-06-10T13:42:59Z","updated_at":"2018-06-10T13:42:59Z","author_association":"MEMBER","body":"I want to keep this issue open, as the performance can be increased for such a case.\r\n\r\nIn the above example,\r\n```python\r\narr.interp(tension=new_tension, resistance=new_resistance)\r\n```\r\nand \r\n```python\r\narr.interp(tension=new_tension).interp(resistance=new_resistance)\r\n```\r\ngives the same result (for 'linear' and 'nearest' methods), but the latter runs much faster.\r\nThis difference looks similar to the difference between our *orthogonal* indexing and *vectorized* indexing.\r\nWe may need *orthogonal* interpolation path, which would significantly increase the performance in some cases."}}
{"actor":{"display_login":"fujiisoup"}}