{"action":"created","comment":{"url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments/58197018","id":58197018,"diff_hunk":"@@ -494,20 +459,87 @@ private void handleRetry(Object cause) {\n         operationService.invocationRegistry.deregister(this);\n \n         if (future.interrupted) {\n-            future.complete(INTERRUPTED_RESPONSE);\n-        } else if (!future.complete(WAIT_RESPONSE)) {\n-            logger.finest(\"Cannot retry \" + toString() + \", because a different response is already set: \" + future.response);\n+            future.complete(INTERRUPTED);\n         } else {\n+            // fast retry for the first few invocations\n             if (invokeCount < MAX_FAST_INVOCATION_COUNT) {\n-                // fast retry for the first few invocations\n                 operationService.asyncExecutor.execute(this);\n             } else {\n-                nodeEngine.getExecutionService().schedule(ASYNC_EXECUTOR, this, tryPauseMillis, MILLISECONDS);\n+                if (invokeCount < MAX_FAST_INVOCATION_COUNT) {\n+                    // fast retry for the first few invocations\n+                    operationService.asyncExecutor.execute(this);\n+                } else {\n+                    nodeEngine.getExecutionService().schedule(ASYNC_EXECUTOR, this, tryPauseMillis, MILLISECONDS);\n+                }\n             }\n         }\n     }\n \n-    boolean checkBackupTimeout(long timeoutMillis) {\n+    /**\n+     * Checks if this Invocation has received a heart-beat in time.\n+     *\n+     * If the response is already set, or if a heart-beat has been received in time, then the false is returned.\n+     *\n+     * If no heart-beat has been received, then the future.set is called with HEARTBEAT_TIMEOUT\n+     * and true is returned.\n+     *\n+     * gets called from the monitor-thread\n+     *\n+     * @return true if there is a timeout detected, false otherwise.\n+     */\n+    boolean detectAndHandleTimeout(long heartbeatTimeoutMillis) {\n+        HeartbeatTimeout heartbeatTimeout = detectTimeout(heartbeatTimeoutMillis);\n+\n+        if (heartbeatTimeout == YES) {\n+            future.complete(HEARTBEAT_TIMEOUT);\n+            return true;\n+        } else {\n+            return false;\n+        }\n+    }\n+\n+    HeartbeatTimeout detectTimeout(long heartbeatTimeoutMillis) {\n+        if (pendingResponse != VOID) {\n+            // if there is a response, then we won't timeout.\n+            return NO__RESPONSE_AVAILABLE;","path":"hazelcast/src/main/java/com/hazelcast/spi/impl/operationservice/impl/Invocation.java","position":231,"original_position":231,"commit_id":"8eab55cde781fc9706ec29b9fdad5cd5edfefc07","original_commit_id":"8eab55cde781fc9706ec29b9fdad5cd5edfefc07","user":{"login":"ahmetmircik","id":2960360,"avatar_url":"https://avatars.githubusercontent.com/u/2960360?v=3","gravatar_id":"","url":"https://api.github.com/users/ahmetmircik","html_url":"https://github.com/ahmetmircik","followers_url":"https://api.github.com/users/ahmetmircik/followers","following_url":"https://api.github.com/users/ahmetmircik/following{/other_user}","gists_url":"https://api.github.com/users/ahmetmircik/gists{/gist_id}","starred_url":"https://api.github.com/users/ahmetmircik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ahmetmircik/subscriptions","organizations_url":"https://api.github.com/users/ahmetmircik/orgs","repos_url":"https://api.github.com/users/ahmetmircik/repos","events_url":"https://api.github.com/users/ahmetmircik/events{/privacy}","received_events_url":"https://api.github.com/users/ahmetmircik/received_events","type":"User","site_admin":false},"body":"This is confusing, even though there is a response (comment says that), method returns `NO__RESPONSE_AVAILABLE`, this lowers code readability and also can cause erroneous cases for the future maintainers due to the naming.\r\n","created_at":"2016-04-01T12:12:59Z","updated_at":"2016-04-01T12:12:59Z","html_url":"https://github.com/hazelcast/hazelcast/pull/7719#discussion_r58197018","pull_request_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719","_links":{"self":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments/58197018"},"html":{"href":"https://github.com/hazelcast/hazelcast/pull/7719#discussion_r58197018"},"pull_request":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719"}}},"pull_request":{"url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719","id":62359238,"html_url":"https://github.com/hazelcast/hazelcast/pull/7719","diff_url":"https://github.com/hazelcast/hazelcast/pull/7719.diff","patch_url":"https://github.com/hazelcast/hazelcast/pull/7719.patch","issue_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/7719","number":7719,"state":"open","locked":false,"title":"Operation timeout logic cleanup","user":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"body":"The IsStillRunning functionality has been replaced by an Operation heartbeat. The OperationService/WaitNotifyService/DistributedExecutorService are frequently scanned for all executing operations to include the call-id. Then a packet is send back including all callid's of the operation each member is executing on another member. Any operation that fails to receive such an operation heartbeat for an extended period will timeout.\r\n\r\nThe old approach was problematic:\r\n- very complicated logic\r\n- unreliable since there is a very short timewindow to ask if an operation is still running. If that timewindow expires (e.g. system has a lot of work to do) then the caller can falsely assume the operation isn't running. The new approach doesn't rely on operations. Operation-control-packet is processed outside of the regular operation-threads.\r\n- a future with get doesn't trigger timeout checking\r\n- Also the logic where the 'CONTINUE_WAIT' on the future was set is removed due to its complexity and problems. E.g.if multiple threads do a get in the same future on a blocking operation, then one of the threads will win, the rest will fail with an OperationTimeoutException.\r\n\r\nfixes:\r\nhttps://github.com/hazelcast/hazelcast/issues/6248\r\nhttps://github.com/hazelcast/hazelcast/issues/4931\r\nhttps://github.com/hazelcast/hazelcast/issues/4410\r\n","created_at":"2016-03-10T09:14:04Z","updated_at":"2016-04-01T12:12:59Z","closed_at":null,"merged_at":null,"merge_commit_sha":"50a5f8e4d5b8c8752d21a38dcc216a9174dd302a","assignee":null,"milestone":{"url":"https://api.github.com/repos/hazelcast/hazelcast/milestones/70","html_url":"https://github.com/hazelcast/hazelcast/milestones/3.7","labels_url":"https://api.github.com/repos/hazelcast/hazelcast/milestones/70/labels","id":1346433,"number":70,"title":"3.7","description":"","creator":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"open_issues":188,"closed_issues":301,"state":"open","created_at":"2015-10-09T12:16:13Z","updated_at":"2016-04-01T11:20:26Z","due_on":"2016-05-02T21:00:00Z","closed_at":null},"commits_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719/commits","review_comments_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719/comments","review_comment_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments{/number}","comments_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/7719/comments","statuses_url":"https://api.github.com/repos/hazelcast/hazelcast/statuses/8eab55cde781fc9706ec29b9fdad5cd5edfefc07","head":{"label":"pveentjer:improvement/3.x/invocation-logic-cleanup","ref":"improvement/3.x/invocation-logic-cleanup","sha":"8eab55cde781fc9706ec29b9fdad5cd5edfefc07","user":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"repo":{"id":10990111,"name":"hazelcast","full_name":"pveentjer/hazelcast","owner":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/pveentjer/hazelcast","description":"Open Source In-Memory Data Grid","fork":true,"url":"https://api.github.com/repos/pveentjer/hazelcast","forks_url":"https://api.github.com/repos/pveentjer/hazelcast/forks","keys_url":"https://api.github.com/repos/pveentjer/hazelcast/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pveentjer/hazelcast/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pveentjer/hazelcast/teams","hooks_url":"https://api.github.com/repos/pveentjer/hazelcast/hooks","issue_events_url":"https://api.github.com/repos/pveentjer/hazelcast/issues/events{/number}","events_url":"https://api.github.com/repos/pveentjer/hazelcast/events","assignees_url":"https://api.github.com/repos/pveentjer/hazelcast/assignees{/user}","branches_url":"https://api.github.com/repos/pveentjer/hazelcast/branches{/branch}","tags_url":"https://api.github.com/repos/pveentjer/hazelcast/tags","blobs_url":"https://api.github.com/repos/pveentjer/hazelcast/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pveentjer/hazelcast/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pveentjer/hazelcast/git/refs{/sha}","trees_url":"https://api.github.com/repos/pveentjer/hazelcast/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pveentjer/hazelcast/statuses/{sha}","languages_url":"https://api.github.com/repos/pveentjer/hazelcast/languages","stargazers_url":"https://api.github.com/repos/pveentjer/hazelcast/stargazers","contributors_url":"https://api.github.com/repos/pveentjer/hazelcast/contributors","subscribers_url":"https://api.github.com/repos/pveentjer/hazelcast/subscribers","subscription_url":"https://api.github.com/repos/pveentjer/hazelcast/subscription","commits_url":"https://api.github.com/repos/pveentjer/hazelcast/commits{/sha}","git_commits_url":"https://api.github.com/repos/pveentjer/hazelcast/git/commits{/sha}","comments_url":"https://api.github.com/repos/pveentjer/hazelcast/comments{/number}","issue_comment_url":"https://api.github.com/repos/pveentjer/hazelcast/issues/comments{/number}","contents_url":"https://api.github.com/repos/pveentjer/hazelcast/contents/{+path}","compare_url":"https://api.github.com/repos/pveentjer/hazelcast/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pveentjer/hazelcast/merges","archive_url":"https://api.github.com/repos/pveentjer/hazelcast/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pveentjer/hazelcast/downloads","issues_url":"https://api.github.com/repos/pveentjer/hazelcast/issues{/number}","pulls_url":"https://api.github.com/repos/pveentjer/hazelcast/pulls{/number}","milestones_url":"https://api.github.com/repos/pveentjer/hazelcast/milestones{/number}","notifications_url":"https://api.github.com/repos/pveentjer/hazelcast/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pveentjer/hazelcast/labels{/name}","releases_url":"https://api.github.com/repos/pveentjer/hazelcast/releases{/id}","deployments_url":"https://api.github.com/repos/pveentjer/hazelcast/deployments","created_at":"2013-06-27T07:15:08Z","updated_at":"2015-04-22T04:00:47Z","pushed_at":"2016-04-01T10:13:02Z","git_url":"git://github.com/pveentjer/hazelcast.git","ssh_url":"git@github.com:pveentjer/hazelcast.git","clone_url":"https://github.com/pveentjer/hazelcast.git","svn_url":"https://github.com/pveentjer/hazelcast","homepage":"www.hazelcast.com","size":95995,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":0,"forks":1,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"hazelcast:master","ref":"master","sha":"1378e2391cd235fa3650e9c29d49b8780f4d326b","user":{"login":"hazelcast","id":1453152,"avatar_url":"https://avatars.githubusercontent.com/u/1453152?v=3","gravatar_id":"","url":"https://api.github.com/users/hazelcast","html_url":"https://github.com/hazelcast","followers_url":"https://api.github.com/users/hazelcast/followers","following_url":"https://api.github.com/users/hazelcast/following{/other_user}","gists_url":"https://api.github.com/users/hazelcast/gists{/gist_id}","starred_url":"https://api.github.com/users/hazelcast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hazelcast/subscriptions","organizations_url":"https://api.github.com/users/hazelcast/orgs","repos_url":"https://api.github.com/users/hazelcast/repos","events_url":"https://api.github.com/users/hazelcast/events{/privacy}","received_events_url":"https://api.github.com/users/hazelcast/received_events","type":"Organization","site_admin":false},"repo":{"id":3786237,"name":"hazelcast","full_name":"hazelcast/hazelcast","owner":{"login":"hazelcast","id":1453152,"avatar_url":"https://avatars.githubusercontent.com/u/1453152?v=3","gravatar_id":"","url":"https://api.github.com/users/hazelcast","html_url":"https://github.com/hazelcast","followers_url":"https://api.github.com/users/hazelcast/followers","following_url":"https://api.github.com/users/hazelcast/following{/other_user}","gists_url":"https://api.github.com/users/hazelcast/gists{/gist_id}","starred_url":"https://api.github.com/users/hazelcast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hazelcast/subscriptions","organizations_url":"https://api.github.com/users/hazelcast/orgs","repos_url":"https://api.github.com/users/hazelcast/repos","events_url":"https://api.github.com/users/hazelcast/events{/privacy}","received_events_url":"https://api.github.com/users/hazelcast/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/hazelcast/hazelcast","description":"Open Source In-Memory Data Grid","fork":false,"url":"https://api.github.com/repos/hazelcast/hazelcast","forks_url":"https://api.github.com/repos/hazelcast/hazelcast/forks","keys_url":"https://api.github.com/repos/hazelcast/hazelcast/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hazelcast/hazelcast/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hazelcast/hazelcast/teams","hooks_url":"https://api.github.com/repos/hazelcast/hazelcast/hooks","issue_events_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/events{/number}","events_url":"https://api.github.com/repos/hazelcast/hazelcast/events","assignees_url":"https://api.github.com/repos/hazelcast/hazelcast/assignees{/user}","branches_url":"https://api.github.com/repos/hazelcast/hazelcast/branches{/branch}","tags_url":"https://api.github.com/repos/hazelcast/hazelcast/tags","blobs_url":"https://api.github.com/repos/hazelcast/hazelcast/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hazelcast/hazelcast/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hazelcast/hazelcast/git/refs{/sha}","trees_url":"https://api.github.com/repos/hazelcast/hazelcast/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hazelcast/hazelcast/statuses/{sha}","languages_url":"https://api.github.com/repos/hazelcast/hazelcast/languages","stargazers_url":"https://api.github.com/repos/hazelcast/hazelcast/stargazers","contributors_url":"https://api.github.com/repos/hazelcast/hazelcast/contributors","subscribers_url":"https://api.github.com/repos/hazelcast/hazelcast/subscribers","subscription_url":"https://api.github.com/repos/hazelcast/hazelcast/subscription","commits_url":"https://api.github.com/repos/hazelcast/hazelcast/commits{/sha}","git_commits_url":"https://api.github.com/repos/hazelcast/hazelcast/git/commits{/sha}","comments_url":"https://api.github.com/repos/hazelcast/hazelcast/comments{/number}","issue_comment_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/comments{/number}","contents_url":"https://api.github.com/repos/hazelcast/hazelcast/contents/{+path}","compare_url":"https://api.github.com/repos/hazelcast/hazelcast/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hazelcast/hazelcast/merges","archive_url":"https://api.github.com/repos/hazelcast/hazelcast/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hazelcast/hazelcast/downloads","issues_url":"https://api.github.com/repos/hazelcast/hazelcast/issues{/number}","pulls_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls{/number}","milestones_url":"https://api.github.com/repos/hazelcast/hazelcast/milestones{/number}","notifications_url":"https://api.github.com/repos/hazelcast/hazelcast/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hazelcast/hazelcast/labels{/name}","releases_url":"https://api.github.com/repos/hazelcast/hazelcast/releases{/id}","deployments_url":"https://api.github.com/repos/hazelcast/hazelcast/deployments","created_at":"2012-03-21T12:25:18Z","updated_at":"2016-04-01T02:50:10Z","pushed_at":"2016-04-01T11:20:28Z","git_url":"git://github.com/hazelcast/hazelcast.git","ssh_url":"git@github.com:hazelcast/hazelcast.git","clone_url":"https://github.com/hazelcast/hazelcast.git","svn_url":"https://github.com/hazelcast/hazelcast","homepage":"www.hazelcast.com","size":109653,"stargazers_count":1497,"watchers_count":1497,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":707,"mirror_url":null,"open_issues_count":596,"forks":707,"open_issues":596,"watchers":1497,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719"},"html":{"href":"https://github.com/hazelcast/hazelcast/pull/7719"},"issue":{"href":"https://api.github.com/repos/hazelcast/hazelcast/issues/7719"},"comments":{"href":"https://api.github.com/repos/hazelcast/hazelcast/issues/7719/comments"},"review_comments":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719/comments"},"review_comment":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/7719/commits"},"statuses":{"href":"https://api.github.com/repos/hazelcast/hazelcast/statuses/8eab55cde781fc9706ec29b9fdad5cd5edfefc07"}}}}