{"action":"created","issue":{"url":"https://api.github.com/repos/matthewgilbert/mapping/issues/4","repository_url":"https://api.github.com/repos/matthewgilbert/mapping","labels_url":"https://api.github.com/repos/matthewgilbert/mapping/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/matthewgilbert/mapping/issues/4/comments","events_url":"https://api.github.com/repos/matthewgilbert/mapping/issues/4/events","html_url":"https://github.com/matthewgilbert/mapping/issues/4","id":304209827,"number":4,"title":"static_transition non performant","user":{"login":"matthewgilbert","id":2401026,"avatar_url":"https://avatars1.githubusercontent.com/u/2401026?v=4","gravatar_id":"","url":"https://api.github.com/users/matthewgilbert","html_url":"https://github.com/matthewgilbert","followers_url":"https://api.github.com/users/matthewgilbert/followers","following_url":"https://api.github.com/users/matthewgilbert/following{/other_user}","gists_url":"https://api.github.com/users/matthewgilbert/gists{/gist_id}","starred_url":"https://api.github.com/users/matthewgilbert/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthewgilbert/subscriptions","organizations_url":"https://api.github.com/users/matthewgilbert/orgs","repos_url":"https://api.github.com/users/matthewgilbert/repos","events_url":"https://api.github.com/users/matthewgilbert/events{/privacy}","received_events_url":"https://api.github.com/users/matthewgilbert/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-03-12T00:29:03Z","updated_at":"2018-03-18T21:27:42Z","closed_at":"2018-03-18T21:27:42Z","author_association":"OWNER","body":"The current implementation of `static_transition` is very non performant. An example below shows the issue.\r\n\r\n```python\r\nimport pandas as pd\r\nimport mapping.mappings\r\n\r\n\r\nmidx = pd.MultiIndex.from_product([[\"A1\", \"A2\", \"A3\"], [\"front\", \"back\"]])\r\ntransition = pd.DataFrame([[1, 0, 1, 0, 1, 0], [0, 1, 0, 1, 0, 1]],\r\n                         index=[-5, -4], columns=midx)\r\n\r\nts = pd.bdate_range(\"2008-01-01\", \"2018-01-01\")\r\ndts = ts[ts.dayofweek == 2]\r\n\r\ncinfo = pd.DataFrame(dts, columns=[\"dates\"])\r\ncinfo.loc[:, \"month\"] = dts.month\r\ncinfo.loc[:, \"year\"] = dts.year\r\ncinfo = cinfo.groupby([\"year\", \"month\"]).nth(2).reset_index()\r\ncode_dict = dict(zip(range(1, 13), \"FGHJKMNQUVXZ\"))\r\ncinfo.loc[:, \"code\"] = cinfo.month.apply(lambda x: code_dict[x])\r\ncinfo.loc[:, \"name\"] = (cinfo.year.apply(str) + \"_\" + cinfo.code +\r\n                        \"_A\")\r\n\r\ncdts = cinfo.dates.copy()\r\ncdts.index = cinfo.name\r\n\r\nts = ts[ts < pd.Timestamp(\"2017-10-01\")]\r\n%timeit mapping.mappings.roller(timestamps=ts, contract_dates=cdts, get_weights=mapping.mappings.static_transition, transition=transition)\r\n\r\n1 loop, best of 3: 11.1 s per loop\r\n```\r\n\r\nThis could likely be sped up by using a `numpy.ndarray` instead of `pandas.DataFrame` since iterative calls to `loc[]` can be quite slow.\r\n\r\nFor reference, a dummy `get_weights()` implementation is timed below\r\n```python\r\ndef dummy_getter(ts, dummy):\r\n    return [(\"A1\", \"2015_Z_A\", 1, ts)]\r\n\r\n%timeit mapping.mappings.roller(timestamps=ts, contract_dates=cdts, get_weights=dummy_getter)\r\n\r\n100 loops, best of 3: 9.87 ms per loop\r\n```"},"comment":{"url":"https://api.github.com/repos/matthewgilbert/mapping/issues/comments/374049615","html_url":"https://github.com/matthewgilbert/mapping/issues/4#issuecomment-374049615","issue_url":"https://api.github.com/repos/matthewgilbert/mapping/issues/4","id":374049615,"user":{"login":"matthewgilbert","id":2401026,"avatar_url":"https://avatars1.githubusercontent.com/u/2401026?v=4","gravatar_id":"","url":"https://api.github.com/users/matthewgilbert","html_url":"https://github.com/matthewgilbert","followers_url":"https://api.github.com/users/matthewgilbert/followers","following_url":"https://api.github.com/users/matthewgilbert/following{/other_user}","gists_url":"https://api.github.com/users/matthewgilbert/gists{/gist_id}","starred_url":"https://api.github.com/users/matthewgilbert/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthewgilbert/subscriptions","organizations_url":"https://api.github.com/users/matthewgilbert/orgs","repos_url":"https://api.github.com/users/matthewgilbert/repos","events_url":"https://api.github.com/users/matthewgilbert/events{/privacy}","received_events_url":"https://api.github.com/users/matthewgilbert/received_events","type":"User","site_admin":false},"created_at":"2018-03-18T21:27:42Z","updated_at":"2018-03-18T21:27:42Z","author_association":"OWNER","body":"This is closed by 46866726bf262cab3183fbbf7bb4f5bcb5f662a3. A large portion of the time taken was coming from calls to `_check_static()` in `static_transition`.  Instead of unnecessarily repeatedly validating the inputs to `static_transition()` when called iteratively from `roller` it is now only validated once at the start."}}
{"actor":{"display_login":"matthewgilbert"}}