{"action":"created","issue":{"url":"https://api.github.com/repos/hazelcast/hazelcast/issues/8531","repository_url":"https://api.github.com/repos/hazelcast/hazelcast","labels_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/8531/labels{/name}","comments_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/8531/comments","events_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/8531/events","html_url":"https://github.com/hazelcast/hazelcast/issues/8531","id":165773110,"number":8531,"title":"TransactionalMap keySet() and values() can see partial commit","user":{"login":"mdogan","id":1201712,"avatar_url":"https://avatars.githubusercontent.com/u/1201712?v=3","gravatar_id":"","url":"https://api.github.com/users/mdogan","html_url":"https://github.com/mdogan","followers_url":"https://api.github.com/users/mdogan/followers","following_url":"https://api.github.com/users/mdogan/following{/other_user}","gists_url":"https://api.github.com/users/mdogan/gists{/gist_id}","starred_url":"https://api.github.com/users/mdogan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdogan/subscriptions","organizations_url":"https://api.github.com/users/mdogan/orgs","repos_url":"https://api.github.com/users/mdogan/repos","events_url":"https://api.github.com/users/mdogan/events{/privacy}","received_events_url":"https://api.github.com/users/mdogan/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/hazelcast/hazelcast/labels/Team:%20Core","name":"Team: Core","color":"109def"},{"url":"https://api.github.com/repos/hazelcast/hazelcast/labels/Type:%20Defect","name":"Type: Defect","color":"eb6420"}],"state":"open","locked":false,"assignee":{"login":"gurbuzali","id":1741680,"avatar_url":"https://avatars.githubusercontent.com/u/1741680?v=3","gravatar_id":"","url":"https://api.github.com/users/gurbuzali","html_url":"https://github.com/gurbuzali","followers_url":"https://api.github.com/users/gurbuzali/followers","following_url":"https://api.github.com/users/gurbuzali/following{/other_user}","gists_url":"https://api.github.com/users/gurbuzali/gists{/gist_id}","starred_url":"https://api.github.com/users/gurbuzali/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gurbuzali/subscriptions","organizations_url":"https://api.github.com/users/gurbuzali/orgs","repos_url":"https://api.github.com/users/gurbuzali/repos","events_url":"https://api.github.com/users/gurbuzali/events{/privacy}","received_events_url":"https://api.github.com/users/gurbuzali/received_events","type":"User","site_admin":false},"assignees":[{"login":"gurbuzali","id":1741680,"avatar_url":"https://avatars.githubusercontent.com/u/1741680?v=3","gravatar_id":"","url":"https://api.github.com/users/gurbuzali","html_url":"https://github.com/gurbuzali","followers_url":"https://api.github.com/users/gurbuzali/followers","following_url":"https://api.github.com/users/gurbuzali/following{/other_user}","gists_url":"https://api.github.com/users/gurbuzali/gists{/gist_id}","starred_url":"https://api.github.com/users/gurbuzali/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gurbuzali/subscriptions","organizations_url":"https://api.github.com/users/gurbuzali/orgs","repos_url":"https://api.github.com/users/gurbuzali/repos","events_url":"https://api.github.com/users/gurbuzali/events{/privacy}","received_events_url":"https://api.github.com/users/gurbuzali/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/hazelcast/hazelcast/milestones/70","html_url":"https://github.com/hazelcast/hazelcast/milestones/3.7","labels_url":"https://api.github.com/repos/hazelcast/hazelcast/milestones/70/labels","id":1346433,"number":70,"title":"3.7","description":"","creator":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"open_issues":44,"closed_issues":855,"state":"open","created_at":"2015-10-09T12:16:13Z","updated_at":"2016-07-18T09:03:43Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2016-07-15T12:07:45Z","updated_at":"2016-07-18T09:31:41Z","closed_at":null,"body":"From _Todd Taomae_\r\nSee post: https://groups.google.com/d/msg/hazelcast/dzXDh7_EU40/HzzJ62NABwAJ\r\n\r\n------------------\r\nConsider a transaction, T1, which calls `TransactionalMap.keySet()` while another transaction, T2, concurrently performs some kind write operation.\r\nSome examples:\r\n- T2 writes 100 entries to an initially empty map.\r\n    - I would expect T1 to find either 0 or 100 entries. However, it is possible for it to find some amount of entries between 0 and 100.\r\n- T2 removes X and puts Y into a map which initially contains X.\r\n    -I would expect T1 to find either X or Y. However, it is also possible for it to find both X and Y or an empty set.\r\n\r\nIn both of these cases I would expect to only get the values that exist before or after the commit. The same applies to the `values()` method as well as their counterparts which take a `Predicate` argument. However, it appears that the read can occur during the middle of the commit.\r\n\r\nBelow is an example based on the second example from above. However, T1 repeatedly performs the `keySet()` and T2 uses an integer as the key and repeatedly removes the previous value and puts the next value (i.e. remove 0 put 1, remove 1 put 2, remove 2 put 3, ...)\r\n\r\n```java\r\nHazelcastInstance hz = Hazelcast.newHazelcastInstance();\r\nhz.getMap(\"map\").put(0, \"foo\");\r\n\r\nExecutorService exec = Executors.newCachedThreadPool();\r\n\r\nAtomicInteger currentKey = new AtomicInteger(0);\r\nfinal int maxKey = 10000;\r\n\r\nFuture<?> txn1 = exec.submit(() -> {\r\n    while (currentKey.get() < maxKey) {\r\n        TransactionContext ctx = hz.newTransactionContext();\r\n        ctx.beginTransaction();\r\n        System.out.println(ctx.getMap(\"map\").keySet());\r\n        ctx.commitTransaction();\r\n    }\r\n});\r\n\r\nFuture<?> txn2 = exec.submit(() -> {\r\n    while (currentKey.get() < maxKey) {\r\n        int key = currentKey.getAndIncrement();\r\n        int nextKey = currentKey.get();\r\n        TransactionContext ctx1 = hz.newTransactionContext();\r\n        ctx1.beginTransaction();\r\n        ctx1.getMap(\"map\").remove(key);\r\n        ctx1.getMap(\"map\").put(nextKey, \"foo\");\r\n        ctx1.commitTransaction();\r\n    }\r\n});\r\n\r\ntxn1.get();\r\ntxn2.get();\r\nexec.shutdown();\r\nexec.awaitTermination(1, TimeUnit.MINUTES);\r\nHazelcast.shutdownAll();\r\n```\r\n\r\nBased on my initial expectations, I would expect that any call to keySet() would return a exactly one key. Based on the two examples I gave at the start, finding an empty set or a set with two consecutive keys would not be surprising. However, during my testing I have found anywhere from 0 to 6 keys (although I assume this could theoretically be arbitrarily large), and if there are multiple keys they can be non-consecutive. I have attached a sample output using Hazelcast 3.6.4.\r\n\r\n[keySet_output.txt](https://github.com/hazelcast/hazelcast/files/365904/keySet_output.txt)\r\n------------------\r\n"},"comment":{"url":"https://api.github.com/repos/hazelcast/hazelcast/issues/comments/233284136","html_url":"https://github.com/hazelcast/hazelcast/issues/8531#issuecomment-233284136","issue_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/8531","id":233284136,"user":{"login":"gurbuzali","id":1741680,"avatar_url":"https://avatars.githubusercontent.com/u/1741680?v=3","gravatar_id":"","url":"https://api.github.com/users/gurbuzali","html_url":"https://github.com/gurbuzali","followers_url":"https://api.github.com/users/gurbuzali/followers","following_url":"https://api.github.com/users/gurbuzali/following{/other_user}","gists_url":"https://api.github.com/users/gurbuzali/gists{/gist_id}","starred_url":"https://api.github.com/users/gurbuzali/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gurbuzali/subscriptions","organizations_url":"https://api.github.com/users/gurbuzali/orgs","repos_url":"https://api.github.com/users/gurbuzali/repos","events_url":"https://api.github.com/users/gurbuzali/events{/privacy}","received_events_url":"https://api.github.com/users/gurbuzali/received_events","type":"User","site_admin":false},"created_at":"2016-07-18T09:31:41Z","updated_at":"2016-07-18T09:31:41Z","body":"The problem here is `keySet` operation spans multiple partitions. For example: \r\n- put-remove transaction not started yet, currentKey(0) resides in partition-5\r\n- keySet operation executes on partition-5 and collects the entry\r\n- put-remove transaction executes, removes entry from partition-5 and puts nextKey(1) to partition-6\r\n- keySet operation executes on partition-6 and collects new entry too\r\n\r\nThere is not a simple solution for this, I'm inclined to postpone this to a later release. wdyt @mdogan ?"}}
{"actor":{"display_login":"gurbuzali"}}