{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/6871","labels_url":"https://api.github.com/repos/pydata/pandas/issues/6871/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/6871/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/6871/events","html_url":"https://github.com/pydata/pandas/issues/6871","id":31326844,"number":6871,"title":"Uncompatibility of pandas.read_pickle with pytz","user":{"login":"AndrewUshakov","id":7263403,"avatar_url":"https://avatars.githubusercontent.com/u/7263403?v=3","gravatar_id":"","url":"https://api.github.com/users/AndrewUshakov","html_url":"https://github.com/AndrewUshakov","followers_url":"https://api.github.com/users/AndrewUshakov/followers","following_url":"https://api.github.com/users/AndrewUshakov/following{/other_user}","gists_url":"https://api.github.com/users/AndrewUshakov/gists{/gist_id}","starred_url":"https://api.github.com/users/AndrewUshakov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndrewUshakov/subscriptions","organizations_url":"https://api.github.com/users/AndrewUshakov/orgs","repos_url":"https://api.github.com/users/AndrewUshakov/repos","events_url":"https://api.github.com/users/AndrewUshakov/events{/privacy}","received_events_url":"https://api.github.com/users/AndrewUshakov/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Bug","name":"Bug","color":"e10c02"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Compat","name":"Compat","color":"0052cc"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/32","html_url":"https://github.com/pydata/pandas/milestones/Next%20Major%20Release","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/32/labels","id":933188,"number":32,"title":"Next Major Release","description":"after 0.16.1 of course","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":970,"closed_issues":117,"state":"open","created_at":"2015-01-13T10:53:19Z","updated_at":"2016-01-27T22:28:31Z","due_on":"2016-01-31T05:00:00Z","closed_at":null},"comments":9,"created_at":"2014-04-11T13:22:52Z","updated_at":"2016-01-28T07:58:39Z","closed_at":null,"body":"Test program below:\r\n```python\r\nimport datetime, pytz, pandas\r\n\r\npandas.show_versions()\r\n\r\ndates = [datetime.datetime(2014,1,1,1,1, tzinfo=pytz.utc),\r\n         datetime.datetime(2014,2,2,2,2, tzinfo=pytz.utc),\r\n         datetime.datetime(2014,3,3,3,3, tzinfo=pytz.utc)]\r\n\r\ns = pandas.Series(dates)\r\ns.to_pickle('series.pickle')\r\n\r\n# import pickle\r\n# with open('series.pickle', 'rb') as f: s1 = pickle.load(f)\r\n\r\ns2 = pandas.read_pickle('series.pickle')\r\n```\r\ncrashes at the last line with long stack trace. If replace \"pytz.utc\" by \"datetime.timezone.utc\", error disappears. Commented code fragment works properly in both cases.\r\n```\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 3.4.0.final.0\r\npython-bits: 64\r\nOS: Windows\r\nOS-release: 7\r\nmachine: AMD64\r\nprocessor: Intel64 Family 6 Model 26 Stepping 5, GenuineIntel\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: None\r\n\r\npandas: 0.13.1\r\nCython: None\r\nnumpy: 1.8.1\r\nscipy: 0.13.3\r\nstatsmodels: 0.5.0\r\nIPython: None\r\nsphinx: 1.2.2\r\npatsy: 0.2.1\r\nscikits.timeseries: None\r\ndateutil: 2.2\r\npytz: 2014.2\r\nbottleneck: None\r\ntables: None\r\nnumexpr: None\r\nmatplotlib: 1.3.1\r\nopenpyxl: None\r\nxlrd: None\r\nxlwt: None\r\nxlsxwriter: None\r\nsqlalchemy: 0.9.4\r\nlxml: 3.3.4\r\nbs4: 4.3.2\r\nhtml5lib: 0.999\r\nbq: None\r\napiclient: None\r\nTraceback (most recent call last):\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\io\\pickle.py\", line 43, in try_read\r\n    return pc.load(fh, encoding=encoding, compat=False)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\compat\\pickle_compat.py\", line 89, in load\r\n    return up.load()\r\n  File \"C:\\Program Files\\Python34\\lib\\pickle.py\", line 1036, in load\r\n    dispatch[key[0]](self)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\compat\\pickle_compat.py\", line 18, in load_reduce\r\n    if type(args[0]) is type:\r\nIndexError: tuple index out of range\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\io\\pickle.py\", line 49, in read_pickle\r\n    return try_read(path)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\io\\pickle.py\", line 46, in try_read\r\n    return pc.load(fh, encoding=encoding, compat=True)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\compat\\pickle_compat.py\", line 89, in load\r\n    return up.load()\r\n  File \"C:\\Program Files\\Python34\\lib\\pickle.py\", line 1036, in load\r\n    dispatch[key[0]](self)\r\n  File \"C:\\Program Files\\Python34\\lib\\pickle.py\", line 1316, in load_newobj_ex\r\n    obj = cls.__new__(cls, *args, **kwargs)\r\nTypeError: Required argument 'shape' (pos 1) not found\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\io\\pickle.py\", line 43, in try_read\r\n    return pc.load(fh, encoding=encoding, compat=False)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\compat\\pickle_compat.py\", line 89, in load\r\n    return up.load()\r\n  File \"C:\\Program Files\\Python34\\lib\\pickle.py\", line 1036, in load\r\n    dispatch[key[0]](self)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\compat\\pickle_compat.py\", line 18, in load_reduce\r\n    if type(args[0]) is type:\r\nIndexError: tuple index out of range\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"H:/My Documents/PyCharm/LFDM/CalcFlowRate/tpicle.py\", line 17, in <module>\r\n    s2 = pandas.read_pickle('series.pickle')\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\io\\pickle.py\", line 52, in read_pickle\r\n    return try_read(path, encoding='latin1')\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\io\\pickle.py\", line 46, in try_read\r\n    return pc.load(fh, encoding=encoding, compat=True)\r\n  File \"C:\\Program Files\\Python34\\lib\\site-packages\\pandas\\compat\\pickle_compat.py\", line 89, in load\r\n    return up.load()\r\n  File \"C:\\Program Files\\Python34\\lib\\pickle.py\", line 1036, in load\r\n    dispatch[key[0]](self)\r\n  File \"C:\\Program Files\\Python34\\lib\\pickle.py\", line 1316, in load_newobj_ex\r\n    obj = cls.__new__(cls, *args, **kwargs)\r\nTypeError: Required argument 'shape' (pos 1) not found\r\n\r\nProcess finished with exit code 1\r\n```\r\n---\r\nBest regards,\r\nAndrew\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/176043180","html_url":"https://github.com/pydata/pandas/issues/6871#issuecomment-176043180","issue_url":"https://api.github.com/repos/pydata/pandas/issues/6871","id":176043180,"user":{"login":"akaihola","id":13725,"avatar_url":"https://avatars.githubusercontent.com/u/13725?v=3","gravatar_id":"","url":"https://api.github.com/users/akaihola","html_url":"https://github.com/akaihola","followers_url":"https://api.github.com/users/akaihola/followers","following_url":"https://api.github.com/users/akaihola/following{/other_user}","gists_url":"https://api.github.com/users/akaihola/gists{/gist_id}","starred_url":"https://api.github.com/users/akaihola/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akaihola/subscriptions","organizations_url":"https://api.github.com/users/akaihola/orgs","repos_url":"https://api.github.com/users/akaihola/repos","events_url":"https://api.github.com/users/akaihola/events{/privacy}","received_events_url":"https://api.github.com/users/akaihola/received_events","type":"User","site_admin":false},"created_at":"2016-01-28T07:58:39Z","updated_at":"2016-01-28T07:58:39Z","body":"We have this problem when writing pickles with Pandas 0.14.1 and reading them with Pandas 0.17.1. The problem occurs if the pickles contain `isodate.UTC` objects.\r\n\r\nThe problem can be reduced to this test case:\r\n\r\n**my_tz.py**:\r\n\r\n    from datetime import tzinfo\r\n\r\n    class MyTz(tzinfo):\r\n        def __init__(self):\r\n            pass\r\n\r\n**write_pickle.py** (run with Pandas 0.14.1):\r\n\r\n    import pandas as pd\r\n    import cPickle\r\n    from my_tz import MyTz\r\n\r\n    data = pd.Series(), MyTz()\r\n    with open('test.pickle', 'wb') as f:\r\n        cPickle.dump(data, f)\r\n\r\n**read_pickle.py** (run with Pandas 0.17.1):\r\n\r\n    import pandas as pd\r\n    pd.read_pickle('test.pickle')\r\n\r\nReading the pickle with `cPickle.load()` works fine, but with `pd.read_pickle()` we get:\r\n\r\n    $ python read_pickle.py\r\n    Traceback (most recent call last):\r\n      File \"read_pickle_test.py\", line 2, in <module>\r\n        pd.read_pickle('test.pickle')\r\n      File \"pandas/io/pickle.py\", line 60, in read_pickle\r\n        return try_read(path)\r\n      File \"pandas/io/pickle.py\", line 57, in try_read\r\n        return pc.load(fh, encoding=encoding, compat=True)\r\n      File \"pandas/compat/pickle_compat.py\", line 116, in load\r\n        return up.load()\r\n      File \"/usr/lib64/python2.7/pickle.py\", line 858, in load\r\n        dispatch[key](self)\r\n      File \"pandas/compat/pickle_compat.py\", line 16, in load_reduce\r\n        if type(args[0]) is type:\r\n    IndexError: tuple index out of range\r\n\r\nI see the following problems in [pandas.compat.pickle_compat.load_reduce()](/pydata/pandas/blob/9bc82438268742575e501d099c6f22ce0a2dcac1/pandas/compat/pickle_compat.py#L13):\r\n\r\n(1) It attemps to access `args[0]` while `args` can be empty.\r\n\r\n    if type(args[0]) is type:\r\n        n = args[0].__name__\r\n\r\n(2) The above code is dead code anyway, since `n` isn't referenced to anywhere else in the function. Removing those two lines fixes the unpickling problem completely.\r\n\r\nNote that the `MyTz` class in the test above does implement a proper `__init__()` method as required in [Python documentation for tzinfo](https://docs.python.org/2.7/library/datetime.html#tzinfo-objects): *\"Special requirement for pickling: A tzinfo subclass must have an `__init__` method that can be called with no arguments, else it can be pickled but possibly not unpickled again. This is a technical requirement that may be relaxed in the future.\"* While `isodate.UTC` violates this requirement, that doesn't seem to be the cause of this problem.\r\n\r\n(3) Also, at the very end of `load_reduce()`:\r\n\r\n    stack[-1] = value\r\n\r\nis code that is never reached, since all previous code branches either return or raise. Also, this line is invalid, since `value` isn't initialized anywhere in the function."}}