{"action":"created","issue":{"url":"https://api.github.com/repos/dask/dask/issues/3376","repository_url":"https://api.github.com/repos/dask/dask","labels_url":"https://api.github.com/repos/dask/dask/issues/3376/labels{/name}","comments_url":"https://api.github.com/repos/dask/dask/issues/3376/comments","events_url":"https://api.github.com/repos/dask/dask/issues/3376/events","html_url":"https://github.com/dask/dask/issues/3376","id":311682438,"number":3376,"title":"Dataframe index is not sorted after `.set_index()`","user":{"login":"bnaul","id":903655,"avatar_url":"https://avatars3.githubusercontent.com/u/903655?v=4","gravatar_id":"","url":"https://api.github.com/users/bnaul","html_url":"https://github.com/bnaul","followers_url":"https://api.github.com/users/bnaul/followers","following_url":"https://api.github.com/users/bnaul/following{/other_user}","gists_url":"https://api.github.com/users/bnaul/gists{/gist_id}","starred_url":"https://api.github.com/users/bnaul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bnaul/subscriptions","organizations_url":"https://api.github.com/users/bnaul/orgs","repos_url":"https://api.github.com/users/bnaul/repos","events_url":"https://api.github.com/users/bnaul/events{/privacy}","received_events_url":"https://api.github.com/users/bnaul/received_events","type":"User","site_admin":false},"labels":[{"id":162914698,"url":"https://api.github.com/repos/dask/dask/labels/good%20first%20issue","name":"good first issue","color":"159818","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":16,"created_at":"2018-04-05T16:29:58Z","updated_at":"2018-04-23T20:07:11Z","closed_at":null,"author_association":"NONE","body":"From the docs+source it appears that the output of `ddf.set_index()` should be sorted by index, but this doesn't seem to be the case.\r\n```\r\nimport dask, dask.dataframe as dd, distributed\r\n\r\nIn [1]: c = distributed.Client('localhost:8786')\r\nIn [2]: ddf = dd.read_csv('gs://<bucket>/2018-01-*.csv.gz', compression='gzip').set_index('uid').compute()\r\nIn [3]: ddf.index.is_monotonic\r\nOut[3]: False\r\nIn [4]: ddf.head().index\r\nOut[4]:\r\nIndex(['00004cd2-8a03-4dc7-85a9-f32ded88d6f6',\r\n       '00004cd2-8a03-4dc7-85a9-f32ded88d6f6',\r\n       '00009051-c962-460b-8090-c1c9d45df180',\r\n       '00009051-c962-460b-8090-c1c9d45df180',\r\n       '00009051-c962-460b-8090-c1c9d45df180'],\r\n      dtype='object', name='uid')\r\n```\r\nThe inputs are randomly ordered, and the output appears to be mostly-but-not-quite sorted; maybe sorted within each partition but not reassembled correctly?"},"comment":{"url":"https://api.github.com/repos/dask/dask/issues/comments/383705570","html_url":"https://github.com/dask/dask/issues/3376#issuecomment-383705570","issue_url":"https://api.github.com/repos/dask/dask/issues/3376","id":383705570,"user":{"login":"bnaul","id":903655,"avatar_url":"https://avatars3.githubusercontent.com/u/903655?v=4","gravatar_id":"","url":"https://api.github.com/users/bnaul","html_url":"https://github.com/bnaul","followers_url":"https://api.github.com/users/bnaul/followers","following_url":"https://api.github.com/users/bnaul/following{/other_user}","gists_url":"https://api.github.com/users/bnaul/gists{/gist_id}","starred_url":"https://api.github.com/users/bnaul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bnaul/subscriptions","organizations_url":"https://api.github.com/users/bnaul/orgs","repos_url":"https://api.github.com/users/bnaul/repos","events_url":"https://api.github.com/users/bnaul/events{/privacy}","received_events_url":"https://api.github.com/users/bnaul/received_events","type":"User","site_admin":false},"created_at":"2018-04-23T20:07:11Z","updated_at":"2018-04-23T20:07:11Z","author_association":"NONE","body":"Yep, `shuffle='tasks'` gives the same incorrect behavior. I tried decreasing the size of the dataframe but with fewer partitions it seems to always end up sorted (4000 breaks, 2000 is OK). Not sure what to make of that 🙃 "}}
{"actor":{"display_login":"bnaul"}}