{"action":"created","issue":{"url":"https://api.github.com/repos/Azure/azure-event-hubs-java/issues/355","repository_url":"https://api.github.com/repos/Azure/azure-event-hubs-java","labels_url":"https://api.github.com/repos/Azure/azure-event-hubs-java/issues/355/labels{/name}","comments_url":"https://api.github.com/repos/Azure/azure-event-hubs-java/issues/355/comments","events_url":"https://api.github.com/repos/Azure/azure-event-hubs-java/issues/355/events","html_url":"https://github.com/Azure/azure-event-hubs-java/pull/355","id":345013065,"node_id":"MDExOlB1bGxSZXF1ZXN0MjA0MjcyOTEx","number":355,"title":"EPH 2.0.1 release -- fix deadlock","user":{"login":"JamesBirdsall","id":10199699,"node_id":"MDQ6VXNlcjEwMTk5Njk5","avatar_url":"https://avatars2.githubusercontent.com/u/10199699?v=4","gravatar_id":"","url":"https://api.github.com/users/JamesBirdsall","html_url":"https://github.com/JamesBirdsall","followers_url":"https://api.github.com/users/JamesBirdsall/followers","following_url":"https://api.github.com/users/JamesBirdsall/following{/other_user}","gists_url":"https://api.github.com/users/JamesBirdsall/gists{/gist_id}","starred_url":"https://api.github.com/users/JamesBirdsall/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JamesBirdsall/subscriptions","organizations_url":"https://api.github.com/users/JamesBirdsall/orgs","repos_url":"https://api.github.com/users/JamesBirdsall/repos","events_url":"https://api.github.com/users/JamesBirdsall/events{/privacy}","received_events_url":"https://api.github.com/users/JamesBirdsall/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-07-26T21:32:04Z","updated_at":"2018-08-02T20:06:07Z","closed_at":"2018-08-02T20:06:07Z","author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/Azure/azure-event-hubs-java/pulls/355","html_url":"https://github.com/Azure/azure-event-hubs-java/pull/355","diff_url":"https://github.com/Azure/azure-event-hubs-java/pull/355.diff","patch_url":"https://github.com/Azure/azure-event-hubs-java/pull/355.patch"},"body":"## Description\r\n\r\nDue to a threading change in client 1.0.2, EPH 2.0.0 has the potential to deadlock due to thread starvation under a particular set of conditions:\r\n* The Event Hub has more partitions than the threadpool has threads\r\n* AND the application is using the default Azure Storage-based checkpoint manager\r\n* AND the IEventProcessor.onEvents implementation checkpoints\r\n* AND a large number of partitions on the same host attempt to checkpoint at almost the same time.\r\n\r\nThe problem: The threading change in client 1.0.2 means that IEventProcessor.onEvents is executing in the same threadpool where the checkpoint manager runs. To ensure ordering of checkpoints, it is important that the onEvents implementation waits for the checkpoint call to return, which means a thread in the threadpool is blocked. If there are too many checkpoint calls in rapid succession, it is possible for all threads in the threadpool to be blocked, leaving no threads available to perform the checkpointing.\r\n\r\nThe fix: This condition occurred because the Azure Storage-base checkpoint manager was attempting to be asynchronous internally, even though the API exposed by Azure Storage is synchronous. Thus, writing one checkpoint required two threads: one running onEvents and blocking, and another to run the task that calls Azure Storage. Since the Azure Storage API is synchronous, this is wasteful at best. Refactored the checkpoint manager to be mostly synchronous and return pre-completed CompletableFutures. This means that onEvents and the Storage calls run on the same thread. That thread still ends up blocked waiting for Storage, but even if every thread in our threadpool ends up blocked that way, the Storage client has its own separate threadpool internally and can always make progress. The Storage calls will always return (successfully or not) and unblock our threads.\r\n\r\nThis checklist is used to make sure that common guidelines for a pull request are followed.\r\n\r\n- [ ] **I have read the [contribution guidelines](./CONTRIBUTING.md).**\r\n- [ ] Title of the pull request is clear and informative.\r\n- [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR.\r\n- [ ] The pull request does not introduce breaking changes (unless a major version change occurs in the assembly and module).\r\n- [ ] If applicable, the public code is properly documented.\r\n- [ ] Pull request includes test coverage for the included changes.\r\n- [ ] The code builds without any errors."},"comment":{"url":"https://api.github.com/repos/Azure/azure-event-hubs-java/issues/comments/410051338","html_url":"https://github.com/Azure/azure-event-hubs-java/pull/355#issuecomment-410051338","issue_url":"https://api.github.com/repos/Azure/azure-event-hubs-java/issues/355","id":410051338,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMDA1MTMzOA==","user":{"login":"JamesBirdsall","id":10199699,"node_id":"MDQ6VXNlcjEwMTk5Njk5","avatar_url":"https://avatars2.githubusercontent.com/u/10199699?v=4","gravatar_id":"","url":"https://api.github.com/users/JamesBirdsall","html_url":"https://github.com/JamesBirdsall","followers_url":"https://api.github.com/users/JamesBirdsall/followers","following_url":"https://api.github.com/users/JamesBirdsall/following{/other_user}","gists_url":"https://api.github.com/users/JamesBirdsall/gists{/gist_id}","starred_url":"https://api.github.com/users/JamesBirdsall/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JamesBirdsall/subscriptions","organizations_url":"https://api.github.com/users/JamesBirdsall/orgs","repos_url":"https://api.github.com/users/JamesBirdsall/repos","events_url":"https://api.github.com/users/JamesBirdsall/events{/privacy}","received_events_url":"https://api.github.com/users/JamesBirdsall/received_events","type":"User","site_admin":false},"created_at":"2018-08-02T20:06:07Z","updated_at":"2018-08-02T20:06:07Z","author_association":"MEMBER","body":"Appveyor not rerunning with changes. Closing, will try again."}}
{"actor":{"display_login":"JamesBirdsall"}}