{"action":"created","issue":{"url":"https://api.github.com/repos/uber/pyro/issues/1673","repository_url":"https://api.github.com/repos/uber/pyro","labels_url":"https://api.github.com/repos/uber/pyro/issues/1673/labels{/name}","comments_url":"https://api.github.com/repos/uber/pyro/issues/1673/comments","events_url":"https://api.github.com/repos/uber/pyro/issues/1673/events","html_url":"https://github.com/uber/pyro/issues/1673","id":391900799,"node_id":"MDU6SXNzdWUzOTE5MDA3OTk=","number":1673,"title":"Pyro NUTS inference slow when compared to the same PyMC3 model","user":{"login":"neerajprad","id":1762463,"node_id":"MDQ6VXNlcjE3NjI0NjM=","avatar_url":"https://avatars3.githubusercontent.com/u/1762463?v=4","gravatar_id":"","url":"https://api.github.com/users/neerajprad","html_url":"https://github.com/neerajprad","followers_url":"https://api.github.com/users/neerajprad/followers","following_url":"https://api.github.com/users/neerajprad/following{/other_user}","gists_url":"https://api.github.com/users/neerajprad/gists{/gist_id}","starred_url":"https://api.github.com/users/neerajprad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/neerajprad/subscriptions","organizations_url":"https://api.github.com/users/neerajprad/orgs","repos_url":"https://api.github.com/users/neerajprad/repos","events_url":"https://api.github.com/users/neerajprad/events{/privacy}","received_events_url":"https://api.github.com/users/neerajprad/received_events","type":"User","site_admin":false},"labels":[{"id":853609549,"node_id":"MDU6TGFiZWw4NTM2MDk1NDk=","url":"https://api.github.com/repos/uber/pyro/labels/profiling","name":"profiling","color":"c2e0c6","default":false},{"id":701053519,"node_id":"MDU6TGFiZWw3MDEwNTM1MTk=","url":"https://api.github.com/repos/uber/pyro/labels/usability","name":"usability","color":"bb8877","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-12-17T21:40:04Z","updated_at":"2018-12-18T18:03:56Z","closed_at":null,"author_association":"COLLABORATOR","body":"For the same models below, the PyMC3 model finishes within a second, whereas the Pyro model has an extremely slow rate of progress. I came across this perf issue while working on a larger model, and isolated it to the example below. I am not sure if this is a perf bug, or whether PyMC3 uses some adaptation tricks, but it is worth looking into and learning from. The cause of the slowdown is the very small step size in Pyro (~10^-9), whereas in PyMC3, the step size throughout is of the order of ~ 10^-3. This is also not a case of bad initialization, the same can be observed for different random seeds.\r\n\r\nAlso refer to #1470. \r\n\r\n```python\r\nimport pymc3 as pm\r\nimport pyro\r\n\r\nimport pyro.distributions as dist\r\nfrom pyro.infer.mcmc import MCMC, NUTS\r\n\r\nimport torch\r\n\r\n\r\ndef pm_model(data):\r\n    with pm.Model() as model:\r\n        alpha = pm.HalfCauchy('alpha', beta=2)\r\n        beta = pm.HalfCauchy('beta', beta=2)\r\n        l = pm.Gamma('l', alpha, beta)\r\n        pm.Poisson('obs', l, observed=data)\r\n    return model\r\n\r\n\r\ndef pyro_model(data):\r\n    alpha = pyro.sample('alpha', dist.HalfCauchy(2.))\r\n    beta = pyro.sample('beta', dist.HalfCauchy(2.))\r\n    l = pyro.sample('l', dist.Gamma(alpha, beta))\r\n    pyro.sample('obs', dist.Poisson(l), obs=data)\r\n\r\n\r\ndef get_samples_pymc(data, num_samples=200, warmup_steps=200):\r\n    data = data.numpy()\r\n    with pm_model(data):\r\n        trace = pm.sample(draws=num_samples, tune=warmup_steps, chains=1)\r\n\r\n\r\ndef get_samples_pyro(data, num_samples=200, warmup_steps=200):\r\n    data = data\r\n    nuts_kernel = NUTS(pyro_model,\r\n                       adapt_step_size=True,\r\n                       adapt_mass_matrix=True,\r\n                       jit_compile=True,\r\n                       full_mass=True)\r\n    mcmc_run = MCMC(nuts_kernel,\r\n                    num_samples=num_samples,\r\n                    warmup_steps=warmup_steps,\r\n                    num_chains=1).run(data)\r\n\r\n\r\ndata = torch.tensor(4805497.)\r\nget_samples_pymc(data)  # finishes in a second.\r\nget_samples_pyro(data)  # orders of magnitude slower.\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/uber/pyro/issues/comments/448313807","html_url":"https://github.com/uber/pyro/issues/1673#issuecomment-448313807","issue_url":"https://api.github.com/repos/uber/pyro/issues/1673","id":448313807,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0ODMxMzgwNw==","user":{"login":"fehiepsi","id":4736342,"node_id":"MDQ6VXNlcjQ3MzYzNDI=","avatar_url":"https://avatars1.githubusercontent.com/u/4736342?v=4","gravatar_id":"","url":"https://api.github.com/users/fehiepsi","html_url":"https://github.com/fehiepsi","followers_url":"https://api.github.com/users/fehiepsi/followers","following_url":"https://api.github.com/users/fehiepsi/following{/other_user}","gists_url":"https://api.github.com/users/fehiepsi/gists{/gist_id}","starred_url":"https://api.github.com/users/fehiepsi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fehiepsi/subscriptions","organizations_url":"https://api.github.com/users/fehiepsi/orgs","repos_url":"https://api.github.com/users/fehiepsi/repos","events_url":"https://api.github.com/users/fehiepsi/events{/privacy}","received_events_url":"https://api.github.com/users/fehiepsi/received_events","type":"User","site_admin":false},"created_at":"2018-12-18T18:03:56Z","updated_at":"2018-12-18T18:03:56Z","author_association":"COLLABORATOR","body":"I just run the program and observe that the small step_size happens at the initial window adaptation, not find_reasonable_step_size. I'll further see how Stan deals with this small step_size problem."}}
{"actor":{"display_login":"fehiepsi"}}