{"action":"created","issue":{"url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1157","repository_url":"https://api.github.com/repos/pymc-devs/pymc3","labels_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1157/labels{/name}","comments_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1157/comments","events_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1157/events","html_url":"https://github.com/pymc-devs/pymc3/issues/1157","id":158416997,"number":1157,"title":"Memory usage with sparse matrices","user":{"login":"igorraush","id":1361385,"avatar_url":"https://avatars.githubusercontent.com/u/1361385?v=3","gravatar_id":"","url":"https://api.github.com/users/igorraush","html_url":"https://github.com/igorraush","followers_url":"https://api.github.com/users/igorraush/followers","following_url":"https://api.github.com/users/igorraush/following{/other_user}","gists_url":"https://api.github.com/users/igorraush/gists{/gist_id}","starred_url":"https://api.github.com/users/igorraush/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/igorraush/subscriptions","organizations_url":"https://api.github.com/users/igorraush/orgs","repos_url":"https://api.github.com/users/igorraush/repos","events_url":"https://api.github.com/users/igorraush/events{/privacy}","received_events_url":"https://api.github.com/users/igorraush/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-06-03T17:36:42Z","updated_at":"2016-06-07T13:16:51Z","closed_at":"2016-06-06T23:57:45Z","body":"I've been trying to implement a very simple logistic regression model, using a sparse design matrix of 0's and 1's (coming from one-hot encoded categorical data). I am seeing large spikes in memory usage. \r\n\r\nHere is a simple example demonstrating the problem (**will briefly use 4GB of memory**):\r\n\r\n```python\r\nfrom __future__ import division, print_function\r\n\r\nimport numpy as np\r\nimport pymc3 as pm\r\nfrom scipy import sparse as sp\r\nfrom scipy.optimize import fmin_l_bfgs_b\r\nfrom scipy.special import expit\r\nfrom theano import sparse as S\r\nnp.random.seed(0)\r\n\r\n# properties of sparse design matrix (taken from the real data)\r\nN = 100000  # number of samples\r\nM = 5000    # number of dimensions\r\nD = 0.002   # matrix density\r\n\r\n# fake data\r\nmu0, sd0 = 0.0, 1.0\r\nw = np.random.normal(mu0, sd0, M)\r\nX = sp.random(N, M, density=D, format='csr', data_rvs=np.ones)\r\ny = np.random.binomial(1, expit(X.dot(w)), N)\r\n\r\n# estimate memory usage\r\nsize = X.data.nbytes + X.indices.nbytes + X.indptr.nbytes + y.nbytes\r\nprint('{:.2f} MB of data'.format(size / 1024 ** 2))\r\n\r\n# model definition\r\nwith pm.Model() as model:\r\n    w = pm.Normal('w', mu0, sd=sd0, shape=M)\r\n\r\n    p = pm.sigmoid(S.dot(X, w.reshape((-1, 1)))).flatten()\r\n    pm.Bernoulli('y', p, observed=y)\r\n\r\n    print(pm.find_MAP(vars=[w], fmin=fmin_l_bfgs_b))\r\n```\r\n\r\nThe sparse matrix occupies roughly 13MB. Incidentally, when it is converted into a dense matrix, it occupies 4GB which corresponds to the peaks in memory usage:\r\n\r\n![memory profile](http://i.stack.imgur.com/g5IqI.png)\r\n\r\nI am aware the problem may well lie entirely in Theano, but I am not sure how to reproduce it outside the PyMC context. Hoping someone can point me in the right direction."},"comment":{"url":"https://api.github.com/repos/pymc-devs/pymc3/issues/comments/224276901","html_url":"https://github.com/pymc-devs/pymc3/issues/1157#issuecomment-224276901","issue_url":"https://api.github.com/repos/pymc-devs/pymc3/issues/1157","id":224276901,"user":{"login":"twiecki","id":674200,"avatar_url":"https://avatars.githubusercontent.com/u/674200?v=3","gravatar_id":"","url":"https://api.github.com/users/twiecki","html_url":"https://github.com/twiecki","followers_url":"https://api.github.com/users/twiecki/followers","following_url":"https://api.github.com/users/twiecki/following{/other_user}","gists_url":"https://api.github.com/users/twiecki/gists{/gist_id}","starred_url":"https://api.github.com/users/twiecki/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twiecki/subscriptions","organizations_url":"https://api.github.com/users/twiecki/orgs","repos_url":"https://api.github.com/users/twiecki/repos","events_url":"https://api.github.com/users/twiecki/events{/privacy}","received_events_url":"https://api.github.com/users/twiecki/received_events","type":"User","site_admin":false},"created_at":"2016-06-07T13:16:51Z","updated_at":"2016-06-07T13:16:51Z","body":"Thanks for reporting back @igorraush. Hopefully Theano can provide a fix soon, sparse support in pymc3 would be a great feature."}}
{"actor":{"display_login":"twiecki"}}