{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/12403","repository_url":"https://api.github.com/repos/pydata/pandas","labels_url":"https://api.github.com/repos/pydata/pandas/issues/12403/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/12403/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/12403/events","html_url":"https://github.com/pydata/pandas/pull/12403","id":135132015,"number":12403,"title":"BUG: Concatenation of DFs with all NaT columns and TZ-aware ones breaks #12396","user":{"login":"hcontrast","id":12200702,"avatar_url":"https://avatars.githubusercontent.com/u/12200702?v=3","gravatar_id":"","url":"https://api.github.com/users/hcontrast","html_url":"https://github.com/hcontrast","followers_url":"https://api.github.com/users/hcontrast/followers","following_url":"https://api.github.com/users/hcontrast/following{/other_user}","gists_url":"https://api.github.com/users/hcontrast/gists{/gist_id}","starred_url":"https://api.github.com/users/hcontrast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hcontrast/subscriptions","organizations_url":"https://api.github.com/users/hcontrast/orgs","repos_url":"https://api.github.com/users/hcontrast/repos","events_url":"https://api.github.com/users/hcontrast/events{/privacy}","received_events_url":"https://api.github.com/users/hcontrast/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Missing-data","name":"Missing-data","color":"d7e102"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Reshaping","name":"Reshaping","color":"02d7e1"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Timezones","name":"Timezones","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/38","html_url":"https://github.com/pydata/pandas/milestones/0.18.1","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/38/labels","id":1570594,"number":38,"title":"0.18.1","description":"after 0.18.0 of course!","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":153,"closed_issues":186,"state":"open","created_at":"2016-02-08T15:29:59Z","updated_at":"2016-04-18T18:19:05Z","due_on":"2016-04-26T04:00:00Z","closed_at":null},"comments":7,"created_at":"2016-02-20T23:06:11Z","updated_at":"2016-04-18T19:47:48Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pydata/pandas/pulls/12403","html_url":"https://github.com/pydata/pandas/pull/12403","diff_url":"https://github.com/pydata/pandas/pull/12403.diff","patch_url":"https://github.com/pydata/pandas/pull/12403.patch"},"body":" - [X] closes #12396\r\n - [X] tests added / passed\r\n - [X] passes ``git diff upstream/master | flake8 --diff``\r\n - [X] whatsnew entry\r\n\r\n* ``concat`` to handle tz-aware all NaT/mixed cases\r\n* added unit tests"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/211548000","html_url":"https://github.com/pydata/pandas/pull/12403#issuecomment-211548000","issue_url":"https://api.github.com/repos/pydata/pandas/issues/12403","id":211548000,"user":{"login":"hcontrast","id":12200702,"avatar_url":"https://avatars.githubusercontent.com/u/12200702?v=3","gravatar_id":"","url":"https://api.github.com/users/hcontrast","html_url":"https://github.com/hcontrast","followers_url":"https://api.github.com/users/hcontrast/followers","following_url":"https://api.github.com/users/hcontrast/following{/other_user}","gists_url":"https://api.github.com/users/hcontrast/gists{/gist_id}","starred_url":"https://api.github.com/users/hcontrast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hcontrast/subscriptions","organizations_url":"https://api.github.com/users/hcontrast/orgs","repos_url":"https://api.github.com/users/hcontrast/repos","events_url":"https://api.github.com/users/hcontrast/events{/privacy}","received_events_url":"https://api.github.com/users/hcontrast/received_events","type":"User","site_admin":false},"created_at":"2016-04-18T19:47:48Z","updated_at":"2016-04-18T19:47:48Z","body":"Updated the tests, but the mixed tz-/non-tz-aware cases are now breaking again:\r\n\r\n```python\r\n======================================================================\r\nERROR: test_concat_NaT_dataframes_all_NaT_axis_0 (pandas.tools.tests.test_merge.TestConcatenate)\r\n----------------------------------------------------------------------\r\nTraceback (most recent call last):\r\n  File \"/.../dev/pandas/pandas/tools/tests/test_merge.py\", line 2606, in test_concat_NaT_dataframes_all_NaT_axis_0\r\n    result = pd.concat([first, second], axis=0)\r\n  File \"/.../dev/pandas/pandas/tools/merge.py\", line 836, in concat\r\n    return op.get_result()\r\n  File \"/.../dev/pandas/pandas/tools/merge.py\", line 1024, in get_result\r\n    concat_axis=self.axis, copy=self.copy)\r\n  File \"/.../dev/pandas/pandas/core/internals.py\", line 4544, in concatenate_block_managers\r\n    for placement, join_units in concat_plan]\r\n  File \"/.../dev/pandas/pandas/core/internals.py\", line 4641, in concatenate_join_units\r\n    for ju in join_units]\r\n  File \"/.../dev/pandas/pandas/core/internals.py\", line 4907, in get_reindexed_values\r\n    missing_arr = np.empty(self.shape, dtype=empty_dtype)\r\nTypeError: data type not understood\r\n```\r\nwhere\r\n```python\r\n# pandas/tools/tests/test_merge.py:2601\r\n\r\n# one side timezone-aware\r\ndtype = DatetimeTZDtype('ns', tz='UTC')\r\nfirst = pd.DataFrame([[pd.NaT], [pd.NaT]], dtype=dtype)\r\n\r\nresult = pd.concat([first, second], axis=0)\r\n```\r\nLooks like accidental/eager upcasting on closer inspection\r\n```\r\nblock:  DatetimeTZBlock: slice(0, 1, 1), 1 x 2, dtype: datetime64[ns, UTC]\r\nis_datetime_tz:  True\r\nempty dtype:  datetime64[ns, UTC]\r\nblock:  DatetimeBlock: slice(0, 1, 1), 1 x 1, dtype: datetime64[ns]\r\nis_datetime_tz:  False\r\nempty dtype:  datetime64[ns, UTC]\r\n```\r\nThe second block is regular datetime but gets passed a tz-aware empty value, hence triggering the wrong case."}}