{"action":"created","issue":{"url":"https://api.github.com/repos/numba/numba/issues/3071","repository_url":"https://api.github.com/repos/numba/numba","labels_url":"https://api.github.com/repos/numba/numba/issues/3071/labels{/name}","comments_url":"https://api.github.com/repos/numba/numba/issues/3071/comments","events_url":"https://api.github.com/repos/numba/numba/issues/3071/events","html_url":"https://github.com/numba/numba/issues/3071","id":337387815,"node_id":"MDU6SXNzdWUzMzczODc4MTU=","number":3071,"title":"Performance issue with lists of arrays?","user":{"login":"leohaim","id":7847768,"node_id":"MDQ6VXNlcjc4NDc3Njg=","avatar_url":"https://avatars3.githubusercontent.com/u/7847768?v=4","gravatar_id":"","url":"https://api.github.com/users/leohaim","html_url":"https://github.com/leohaim","followers_url":"https://api.github.com/users/leohaim/followers","following_url":"https://api.github.com/users/leohaim/following{/other_user}","gists_url":"https://api.github.com/users/leohaim/gists{/gist_id}","starred_url":"https://api.github.com/users/leohaim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leohaim/subscriptions","organizations_url":"https://api.github.com/users/leohaim/orgs","repos_url":"https://api.github.com/users/leohaim/repos","events_url":"https://api.github.com/users/leohaim/events{/privacy}","received_events_url":"https://api.github.com/users/leohaim/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-07-02T06:41:01Z","updated_at":"2018-07-30T07:39:32Z","closed_at":null,"author_association":"NONE","body":"Hello,\r\nit's great to have lists of numpy arrays now supported in Numba!\r\n\r\nI experimented a bit and found that Numba is slower than Numpy for short lists of large arrays but\r\nfaster than Numpy for long lists of small arrays. The timings are on a Xeon E5 with Numba 0.39RC1 installed with Conda for python 2.7:\r\n\r\n```\r\nsrvx1{~leo/python}165 %python arraylist2.py\r\nshort list of large arrays:\r\nResult n=50: 147563.124\r\nNumpy Time: 0.740\r\nResult n=50: 147563.124\r\nNumba Time: 2.120\r\n\r\nlong list of small arrays:\r\nResult n=500000: 14.441\r\nNumpy Time: 7.186\r\nResult n =500000: 14.441\r\nNumba Time: 1.570\r\n```\r\n\r\nWhat is the reason for Numba being slower than Numpy in the first example? My use case is more like the first example. \r\n\r\nThe reproducer, which is really just for testing:\r\n\r\n```python\r\n# -*- coding: utf-8 -*-\r\n\"\"\"\r\nCreated on Thu Jun 28 18:45:46 2018\r\n\r\n@author: Leo\r\n\"\"\"\r\n\r\nimport numpy\r\nfrom numba import *\r\nimport time\r\n\r\n\r\ndef uselist(al):\r\n    lens=numpy.empty(len(al))\r\n    for i in range(len(al)):\r\n        al[i][:,:]*=i\r\n        lens[i]=al[i].shape[1]\r\n        \r\n    return numpy.std(lens)\r\n\r\ndef genlist(lens):\r\n    al=[]\r\n    for i in range(lens.shape[0]):\r\n        al.append(numpy.zeros((16,lens[i]))+1.0)\r\n    \r\n    return uselist(al)\r\n\r\n@njit\r\ndef nuselist(al):\r\n    lens=numpy.empty(len(al))\r\n    for i in range(len(al)):\r\n        al[i][:,:]*=i\r\n        lens[i]=al[i].shape[1]\r\n        \r\n    return numpy.std(lens)\r\n\r\n@njit\r\ndef ngenlist(lens):\r\n    al=[]\r\n    for i in range(lens.shape[0]):\r\n        al.append(numpy.zeros((16,lens[i]))+1.0)\r\n    \r\n    return nuselist(al)\r\n\r\n\r\nprint ('short list of large arrays:')\r\n\r\nn=50\r\nm=500000\r\nlens=numpy.random.randint(m,size=n)\r\nal=ngenlist(lens)\r\n\r\nfor it in range(1):\r\n    t=time.time()\r\n    print('Result n={}: {:5.3f}'.format(n, genlist(lens)))\r\n    print('Numpy Time: {:5.3f}'.format(time.time()-t))\r\n\r\nfor it in range(1):\r\n    t=time.time()\r\n    print('Result n={}: {:5.3f}'.format(n, ngenlist(lens)))\r\n    print('Numba Time: {:5.3f}'.format(time.time()-t))\r\n\r\nprint(\"\")\r\nprint ('long list of small arrays:')\r\n\r\nn=500000\r\nm=50\r\nlens=numpy.random.randint(m,size=n)\r\n\r\nfor it in range(1):\r\n    t=time.time()\r\n    print('Result n={}: {:5.3f}'.format(n,genlist(lens)))\r\n    print('Numpy Time: {:5.3f}'.format(time.time()-t))\r\n\r\nfor it in range(1):\r\n    t=time.time()\r\n    print('Result n ={}: {:5.3f}'.format(n,ngenlist(lens)))\r\n    print('Numba Time: {:5.3f}'.format(time.time()-t))\r\n\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/numba/numba/issues/comments/408774831","html_url":"https://github.com/numba/numba/issues/3071#issuecomment-408774831","issue_url":"https://api.github.com/repos/numba/numba/issues/3071","id":408774831,"node_id":"MDEyOklzc3VlQ29tbWVudDQwODc3NDgzMQ==","user":{"login":"bordingj","id":7777521,"node_id":"MDQ6VXNlcjc3Nzc1MjE=","avatar_url":"https://avatars1.githubusercontent.com/u/7777521?v=4","gravatar_id":"","url":"https://api.github.com/users/bordingj","html_url":"https://github.com/bordingj","followers_url":"https://api.github.com/users/bordingj/followers","following_url":"https://api.github.com/users/bordingj/following{/other_user}","gists_url":"https://api.github.com/users/bordingj/gists{/gist_id}","starred_url":"https://api.github.com/users/bordingj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bordingj/subscriptions","organizations_url":"https://api.github.com/users/bordingj/orgs","repos_url":"https://api.github.com/users/bordingj/repos","events_url":"https://api.github.com/users/bordingj/events{/privacy}","received_events_url":"https://api.github.com/users/bordingj/received_events","type":"User","site_admin":false},"created_at":"2018-07-30T07:39:32Z","updated_at":"2018-07-30T07:39:32Z","author_association":"NONE","body":"I am experiencing the same thing. \r\n\r\nHere is another minimal example:\r\n\r\n\r\n```python\r\ndef sum_list(l):\r\n    s = np.zeros_like(l[0])\r\n    for i in range(1,len(l)):\r\n        s += l[i]\r\n    return s\r\n\r\nsum_list_nb = nb.njit(sum_list)\r\n\r\nl = [np.random.rand(10),]*100\r\n\r\nsum_list_nb(l)\r\n```\r\n\r\nGives me\r\n\r\n```python\r\nIn[33]: %timeit sum_list(l)\r\n71.1 µs ± 72.1 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)\r\nIn[34]: %timeit sum_list_nb(l)\r\n906 µs ± 2.57 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\r\n```"}}
{"actor":{"display_login":"bordingj"}}