{"action":"reopened","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/4264","repository_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/4264/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/4264/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/4264/events","html_url":"https://github.com/GoogleCloudPlatform/google-cloud-python/issues/4264","id":268854436,"number":4264,"title":"Datastore: receiving duplicate entities during query iteration","user":{"login":"berlincode","id":9403685,"avatar_url":"https://avatars0.githubusercontent.com/u/9403685?v=4","gravatar_id":"","url":"https://api.github.com/users/berlincode","html_url":"https://github.com/berlincode","followers_url":"https://api.github.com/users/berlincode/followers","following_url":"https://api.github.com/users/berlincode/following{/other_user}","gists_url":"https://api.github.com/users/berlincode/gists{/gist_id}","starred_url":"https://api.github.com/users/berlincode/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/berlincode/subscriptions","organizations_url":"https://api.github.com/users/berlincode/orgs","repos_url":"https://api.github.com/users/berlincode/repos","events_url":"https://api.github.com/users/berlincode/events{/privacy}","received_events_url":"https://api.github.com/users/berlincode/received_events","type":"User","site_admin":false},"labels":[{"id":77676109,"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/labels/api:%20datastore","name":"api: datastore","color":"f8f8f8","default":false},{"id":544604243,"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/labels/priority:%20p2","name":"priority: p2","color":"fef2c0","default":false},{"id":77438580,"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/labels/type:%20bug","name":"type: bug","color":"d93f0b","default":false}],"state":"open","locked":false,"assignee":{"login":"tseaver","id":242750,"avatar_url":"https://avatars2.githubusercontent.com/u/242750?v=4","gravatar_id":"","url":"https://api.github.com/users/tseaver","html_url":"https://github.com/tseaver","followers_url":"https://api.github.com/users/tseaver/followers","following_url":"https://api.github.com/users/tseaver/following{/other_user}","gists_url":"https://api.github.com/users/tseaver/gists{/gist_id}","starred_url":"https://api.github.com/users/tseaver/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tseaver/subscriptions","organizations_url":"https://api.github.com/users/tseaver/orgs","repos_url":"https://api.github.com/users/tseaver/repos","events_url":"https://api.github.com/users/tseaver/events{/privacy}","received_events_url":"https://api.github.com/users/tseaver/received_events","type":"User","site_admin":false},"assignees":[{"login":"tseaver","id":242750,"avatar_url":"https://avatars2.githubusercontent.com/u/242750?v=4","gravatar_id":"","url":"https://api.github.com/users/tseaver","html_url":"https://github.com/tseaver","followers_url":"https://api.github.com/users/tseaver/followers","following_url":"https://api.github.com/users/tseaver/following{/other_user}","gists_url":"https://api.github.com/users/tseaver/gists{/gist_id}","starred_url":"https://api.github.com/users/tseaver/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tseaver/subscriptions","organizations_url":"https://api.github.com/users/tseaver/orgs","repos_url":"https://api.github.com/users/tseaver/repos","events_url":"https://api.github.com/users/tseaver/events{/privacy}","received_events_url":"https://api.github.com/users/tseaver/received_events","type":"User","site_admin":false}],"milestone":null,"comments":8,"created_at":"2017-10-26T17:57:40Z","updated_at":"2018-02-22T22:51:32Z","closed_at":null,"author_association":"NONE","body":"Hi.\r\n\r\nI'm using google-cloud-datastore==1.3.0 from pypi.\r\n\r\nWhen iterating over a query iterator the google-cloud-datastore sometimes returns the same entitiy twice.\r\n\r\nI've had a look at the source and it seems that the last entity of a fetched page is the same as the first entity of the next page.\r\n\r\nIterating of pages and printing each enitity id makes the problem obvious.\r\n```python\r\nquery_iter = qry.fetch()\r\npages = query_iter.pages\r\nwhile True:\r\n    page = next(pages)\r\n    if not page:\r\n        return\r\n\r\n    for ent in page:\r\n        print(ent.key.id)\r\n\r\n    print(\"-- new page --\")\r\n```\r\n\r\nThe output looks like this:\r\n```\r\n6438744212111360\r\n5774037759295488\r\n5226622299930624\r\n6390563805855744\r\n5537056630505472\r\n5007404484788224\r\n5289011263307776\r\n5805078561685504\r\n-- new page --\r\n5805078561685504\r\n4785039263924224\r\n4655690686660608\r\n5253444773347328\r\n5481464721309696\r\n```\r\n\r\nAs you can see the entity 5805078561685504 is returned twice.\r\n\r\nDumping the protocol buffer shows that the 'cursor'-value of the last entity and is identical to the 'end_cursor'-value. Here are the last lines of the dump:```\r\n\r\n```\r\n      }\r\n    }\r\n  }\r\n  cursor: \"\\n\\346\\002\\n\\260\\002\\n\\003ka2\\022\\250\\002\\032\\245\\002...\"\r\n  version: 1507954620963000\r\n}\r\nend_cursor: \"\\n\\346\\002\\n\\260\\002\\n\\003ka2\\022\\250\\002\\032\\245\\002...\"\r\nmore_results: NOT_FINISHED\r\n```\r\n\r\nMaybe you can help."}}
{"actor":{"display_login":"tseaver"}}