{"action":"created","issue":{"url":"https://api.github.com/repos/pywr/pywr/issues/338","repository_url":"https://api.github.com/repos/pywr/pywr","labels_url":"https://api.github.com/repos/pywr/pywr/issues/338/labels{/name}","comments_url":"https://api.github.com/repos/pywr/pywr/issues/338/comments","events_url":"https://api.github.com/repos/pywr/pywr/issues/338/events","html_url":"https://github.com/pywr/pywr/issues/338","id":194769245,"number":338,"title":"Wrapped cython calls in GLPK solver","user":{"login":"jetuk","id":4604654,"avatar_url":"https://avatars.githubusercontent.com/u/4604654?v=3","gravatar_id":"","url":"https://api.github.com/users/jetuk","html_url":"https://github.com/jetuk","followers_url":"https://api.github.com/users/jetuk/followers","following_url":"https://api.github.com/users/jetuk/following{/other_user}","gists_url":"https://api.github.com/users/jetuk/gists{/gist_id}","starred_url":"https://api.github.com/users/jetuk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jetuk/subscriptions","organizations_url":"https://api.github.com/users/jetuk/orgs","repos_url":"https://api.github.com/users/jetuk/repos","events_url":"https://api.github.com/users/jetuk/events{/privacy}","received_events_url":"https://api.github.com/users/jetuk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-12-10T13:59:00Z","updated_at":"2016-12-16T08:02:28Z","closed_at":null,"body":"Here's some output from a recent profile of a big model. The 53s of the solve is in the main solver function. 39s is inside the function, but 14s (~25%) is in calls to other functions.\r\n\r\n```\r\n ncalls  tottime  percall  cumtime  percall filename:lineno(function)\r\n10600   38.932    0.004   53.300    0.005 pywr\\solvers\\cython_glpk.pyx:353(_solve_scenario)\r\n```\r\n\r\nLook at the calls from `_solve_scenario` we can see a bunch are \"wrapped\". This means they are not C calls, but are dropping to Python space instead. This happens to two functions,\r\n\r\n1. `commit` is [called from an untyped object.](https://github.com/pywr/pywr/blob/master/pywr/solvers/cython_glpk.pyx#L472)\r\n2. `get_cost` is called from a typed object, but that type is `AbstractNode` which doesn't have a cdef'd `get_cost`. This is only defined from `Node` upwards.\r\n\r\nFixing these should improve performance a little.\r\n\r\n```\r\n   Ordered by: internal time\r\n   List reduced from 315 to 1 due to restriction <'_solve_scenario'>\r\n\r\nFunction                                           called...\r\n                                                       ncalls  tottime  cumtime\r\npywr\\solvers\\cython_glpk.pyx:353(_solve_scenario)  ->  212000    0.017    0.017  pywr\\_core.pyx:164(__get__)\r\n                                                      4494400    1.053    2.007  pywr\\_core.pyx:292(commit (wrapper))\r\n                                                      4293000    1.024    1.346  pywr\\_core.pyx:352(get_cost (wrapper))\r\n                                                      4346000    1.387    1.557  pywr\\_core.pyx:377(get_min_flow)\r\n                                                      4346000    1.561    5.756  pywr\\_core.pyx:404(get_max_flow)\r\n                                                        95400    0.038    0.134  pywr\\_core.pyx:598(commit (wrapper))\r\n                                                        95400    0.037    0.560  pywr\\_core.pyx:602(get_cost (wrapper))\r\n                                                        95400    0.031    0.136  pywr\\_core.pyx:607(commit (wrapper))\r\n                                                        95400    0.034    0.947  pywr\\_core.pyx:611(get_cost (wrapper))\r\n                                                       106000    0.032    0.474  pywr\\_core.pyx:676(get_cost (wrapper))\r\n                                                       106000    0.040    0.040  pywr\\_core.pyx:703(get_min_volume)\r\n                                                       106000    0.037    0.037  pywr\\_core.pyx:721(get_max_volume)\r\n                                                      4452000    0.412    0.412  pywr\\solvers\\glpk.pxi:13(constraint_type)\r\n                                                      8692000    0.761    0.761  pywr\\solvers\\glpk.pxi:35(inf_to_dbl_max)\r\n                                                        42400    0.056    0.090  stringsource:643(memoryview_cwrapper)\r\n                                                        42400    0.013    0.013  stringsource:649(memoryview_check)\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/pywr/pywr/issues/comments/267538826","html_url":"https://github.com/pywr/pywr/issues/338#issuecomment-267538826","issue_url":"https://api.github.com/repos/pywr/pywr/issues/338","id":267538826,"user":{"login":"snorfalorpagus","id":3883947,"avatar_url":"https://avatars.githubusercontent.com/u/3883947?v=3","gravatar_id":"","url":"https://api.github.com/users/snorfalorpagus","html_url":"https://github.com/snorfalorpagus","followers_url":"https://api.github.com/users/snorfalorpagus/followers","following_url":"https://api.github.com/users/snorfalorpagus/following{/other_user}","gists_url":"https://api.github.com/users/snorfalorpagus/gists{/gist_id}","starred_url":"https://api.github.com/users/snorfalorpagus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/snorfalorpagus/subscriptions","organizations_url":"https://api.github.com/users/snorfalorpagus/orgs","repos_url":"https://api.github.com/users/snorfalorpagus/repos","events_url":"https://api.github.com/users/snorfalorpagus/events{/privacy}","received_events_url":"https://api.github.com/users/snorfalorpagus/received_events","type":"User","site_admin":false},"created_at":"2016-12-16T08:02:28Z","updated_at":"2016-12-16T08:02:28Z","body":"My reason for not putting `get_cost` on `AbstractNode` initially is that it's not valid for all nodes to have costs (at least currently)."}}
{"actor":{"display_login":"snorfalorpagus"}}