{"action":"created","issue":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/24198","repository_url":"https://api.github.com/repos/pandas-dev/pandas","labels_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24198/labels{/name}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24198/comments","events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24198/events","html_url":"https://github.com/pandas-dev/pandas/issues/24198","id":389206725,"node_id":"MDU6SXNzdWUzODkyMDY3MjU=","number":24198,"title":"Lost timezone after groupby transform","user":{"login":"pocin","id":8642344,"node_id":"MDQ6VXNlcjg2NDIzNDQ=","avatar_url":"https://avatars3.githubusercontent.com/u/8642344?v=4","gravatar_id":"","url":"https://api.github.com/users/pocin","html_url":"https://github.com/pocin","followers_url":"https://api.github.com/users/pocin/followers","following_url":"https://api.github.com/users/pocin/following{/other_user}","gists_url":"https://api.github.com/users/pocin/gists{/gist_id}","starred_url":"https://api.github.com/users/pocin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pocin/subscriptions","organizations_url":"https://api.github.com/users/pocin/orgs","repos_url":"https://api.github.com/users/pocin/repos","events_url":"https://api.github.com/users/pocin/events{/privacy}","received_events_url":"https://api.github.com/users/pocin/received_events","type":"User","site_admin":false},"labels":[{"id":233160,"node_id":"MDU6TGFiZWwyMzMxNjA=","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Groupby","name":"Groupby","color":"729FCF","default":false},{"id":60458168,"node_id":"MDU6TGFiZWw2MDQ1ODE2OA==","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Timezones","name":"Timezones","color":"5319e7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-12-10T09:18:14Z","updated_at":"2018-12-10T18:12:18Z","closed_at":null,"author_association":"NONE","body":"#### Code Sample, a copy-pastable example if possible\r\n\r\n```python\r\nimport pandas as pd\r\n\r\ndf = pd.DataFrame({'end_time': [pd.to_datetime('now', utc=True).tz_convert('Asia/Singapore')],\r\n                  'id': [1]})\r\n\r\ndf['max_end_time'] = df.groupby('id').end_time.transform(max)\r\n```\r\n\r\n`df.info()` shows \r\n\r\n```\r\n<class 'pandas.core.frame.DataFrame'>\r\nRangeIndex: 1 entries, 0 to 0\r\nData columns (total 3 columns):\r\nend_time        1 non-null datetime64[ns, Asia/Singapore]\r\nid              1 non-null int64\r\nmax_end_time    1 non-null datetime64[ns]\r\ndtypes: datetime64[ns, Asia/Singapore](1), datetime64[ns](1), int64(1)\r\nmemory usage: 104.0 bytes\r\n```\r\n\r\n`df.to_dict()` shows\r\n```python\r\n{'end_time': {0: Timestamp('2018-12-10 17:08:52.630644+0800', tz='Asia/Singapore')},\r\n 'id': {0: 1},\r\n 'max_end_time': {0: Timestamp('2018-12-10 09:08:52.630644')}}\r\n```\r\n#### Problem description\r\n\r\nThe timezone is dropped silently and timestamp converted to UTC after groupby - transform operation on tz aware datetime column\r\n\r\n#### Expected Output\r\n`assert df['end_time'] == df['max_end_time']`\r\n\r\n#### Output of ``pd.show_versions()``\r\n\r\n<details>\r\n```\r\nINSTALLED VERSIONS\r\n------------------\r\ncommit: None\r\npython: 3.7.1.final.0\r\npython-bits: 64\r\nOS: Linux\r\nOS-release: 4.9.85-38.58.amzn1.x86_64\r\nmachine: x86_64\r\nprocessor: \r\nbyteorder: little\r\nLC_ALL: en_US.UTF-8\r\nLANG: en_US.UTF-8\r\nLOCALE: en_US.UTF-8\r\n\r\npandas: 0.23.4\r\npytest: 3.10.0\r\npip: 18.1\r\nsetuptools: 40.5.0\r\nCython: 0.29\r\nnumpy: 1.15.4\r\nscipy: 1.1.0\r\npyarrow: None\r\nxarray: None\r\nIPython: 7.1.1\r\nsphinx: None\r\npatsy: 0.5.1\r\ndateutil: 2.7.5\r\npytz: 2018.7\r\nblosc: None\r\nbottleneck: None\r\ntables: None\r\nnumexpr: 2.6.8\r\nfeather: None\r\nmatplotlib: 3.0.1\r\nopenpyxl: None\r\nxlrd: 1.1.0\r\nxlwt: None\r\nxlsxwriter: None\r\nlxml: None\r\nbs4: 4.6.3\r\nhtml5lib: None\r\nsqlalchemy: 1.2.13\r\npymysql: None\r\npsycopg2: None\r\njinja2: 2.10\r\ns3fs: None\r\nfastparquet: None\r\npandas_gbq: None\r\npandas_datareader: None\r\n```\r\n</details>\r\n"},"comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments/445915136","html_url":"https://github.com/pandas-dev/pandas/issues/24198#issuecomment-445915136","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/24198","id":445915136,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NTkxNTEzNg==","user":{"login":"mroeschke","id":10647082,"node_id":"MDQ6VXNlcjEwNjQ3MDgy","avatar_url":"https://avatars0.githubusercontent.com/u/10647082?v=4","gravatar_id":"","url":"https://api.github.com/users/mroeschke","html_url":"https://github.com/mroeschke","followers_url":"https://api.github.com/users/mroeschke/followers","following_url":"https://api.github.com/users/mroeschke/following{/other_user}","gists_url":"https://api.github.com/users/mroeschke/gists{/gist_id}","starred_url":"https://api.github.com/users/mroeschke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mroeschke/subscriptions","organizations_url":"https://api.github.com/users/mroeschke/orgs","repos_url":"https://api.github.com/users/mroeschke/repos","events_url":"https://api.github.com/users/mroeschke/events{/privacy}","received_events_url":"https://api.github.com/users/mroeschke/received_events","type":"User","site_admin":false},"created_at":"2018-12-10T18:12:18Z","updated_at":"2018-12-10T18:12:18Z","author_association":"MEMBER","body":"@WillAyd Since it appears that the groupby routines convert to UTC first before running the aggregates, should be as straightforward to ensure that tz-aware columns undergo a `tz_localize('UTC').tz_convert(tz)` after the cython routine. Shouldn't need to worry about ambiguous/nonexistent times. "}}
{"actor":{"display_login":"mroeschke"}}