{"action":"created","issue":{"url":"https://api.github.com/repos/minio/minio/issues/3274","repository_url":"https://api.github.com/repos/minio/minio","labels_url":"https://api.github.com/repos/minio/minio/issues/3274/labels{/name}","comments_url":"https://api.github.com/repos/minio/minio/issues/3274/comments","events_url":"https://api.github.com/repos/minio/minio/issues/3274/events","html_url":"https://github.com/minio/minio/issues/3274","id":189951586,"number":3274,"title":"[distributed]  `exec-concurrent TestPutObjectReadAt` test hangs with leaked go routines.","user":{"login":"hackintoshrao","id":2609511,"avatar_url":"https://avatars.githubusercontent.com/u/2609511?v=3","gravatar_id":"","url":"https://api.github.com/users/hackintoshrao","html_url":"https://github.com/hackintoshrao","followers_url":"https://api.github.com/users/hackintoshrao/followers","following_url":"https://api.github.com/users/hackintoshrao/following{/other_user}","gists_url":"https://api.github.com/users/hackintoshrao/gists{/gist_id}","starred_url":"https://api.github.com/users/hackintoshrao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hackintoshrao/subscriptions","organizations_url":"https://api.github.com/users/hackintoshrao/orgs","repos_url":"https://api.github.com/users/hackintoshrao/repos","events_url":"https://api.github.com/users/hackintoshrao/events{/privacy}","received_events_url":"https://api.github.com/users/hackintoshrao/received_events","type":"User","site_admin":false},"labels":[{"id":166285390,"url":"https://api.github.com/repos/minio/minio/labels/bug","name":"bug","color":"fc2929","default":true}],"state":"open","locked":false,"assignee":{"login":"krisis","id":522593,"avatar_url":"https://avatars.githubusercontent.com/u/522593?v=3","gravatar_id":"","url":"https://api.github.com/users/krisis","html_url":"https://github.com/krisis","followers_url":"https://api.github.com/users/krisis/followers","following_url":"https://api.github.com/users/krisis/following{/other_user}","gists_url":"https://api.github.com/users/krisis/gists{/gist_id}","starred_url":"https://api.github.com/users/krisis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krisis/subscriptions","organizations_url":"https://api.github.com/users/krisis/orgs","repos_url":"https://api.github.com/users/krisis/repos","events_url":"https://api.github.com/users/krisis/events{/privacy}","received_events_url":"https://api.github.com/users/krisis/received_events","type":"User","site_admin":false},"assignees":[{"login":"krisis","id":522593,"avatar_url":"https://avatars.githubusercontent.com/u/522593?v=3","gravatar_id":"","url":"https://api.github.com/users/krisis","html_url":"https://github.com/krisis","followers_url":"https://api.github.com/users/krisis/followers","following_url":"https://api.github.com/users/krisis/following{/other_user}","gists_url":"https://api.github.com/users/krisis/gists{/gist_id}","starred_url":"https://api.github.com/users/krisis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krisis/subscriptions","organizations_url":"https://api.github.com/users/krisis/orgs","repos_url":"https://api.github.com/users/krisis/repos","events_url":"https://api.github.com/users/krisis/events{/privacy}","received_events_url":"https://api.github.com/users/krisis/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/minio/minio/milestones/11","html_url":"https://github.com/minio/minio/milestones/Distributed%20Server%20Release%20-%20Hardening%20(Phase%202)","labels_url":"https://api.github.com/repos/minio/minio/milestones/11/labels","id":1990140,"number":11,"title":"Distributed Server Release - Hardening (Phase 2)","description":"","creator":{"login":"harshavardhana","id":622699,"avatar_url":"https://avatars.githubusercontent.com/u/622699?v=3","gravatar_id":"","url":"https://api.github.com/users/harshavardhana","html_url":"https://github.com/harshavardhana","followers_url":"https://api.github.com/users/harshavardhana/followers","following_url":"https://api.github.com/users/harshavardhana/following{/other_user}","gists_url":"https://api.github.com/users/harshavardhana/gists{/gist_id}","starred_url":"https://api.github.com/users/harshavardhana/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/harshavardhana/subscriptions","organizations_url":"https://api.github.com/users/harshavardhana/orgs","repos_url":"https://api.github.com/users/harshavardhana/repos","events_url":"https://api.github.com/users/harshavardhana/events{/privacy}","received_events_url":"https://api.github.com/users/harshavardhana/received_events","type":"User","site_admin":false},"open_issues":14,"closed_issues":190,"state":"open","created_at":"2016-09-08T00:59:07Z","updated_at":"2016-11-18T20:38:36Z","due_on":"2016-11-20T00:00:00Z","closed_at":null},"comments":9,"created_at":"2016-11-17T06:26:25Z","updated_at":"2016-11-19T12:00:42Z","closed_at":null,"body":"<!--- Provide a general summary of the issue in the Title above -->\r\n - `exec-concurrent TestPutObjectReadAt` test hangs on the distributed setup.\r\n- Witnessed a severe Go routine leakage (Closely related to https://github.com/minio/minio/issues/3260)\r\n## Expected Behavior\r\n<!--- If you're describing a bug, tell us what should happen -->\r\n<!--- If you're suggesting a change/improvement, tell us how it should work -->\r\n- Test to pass.\r\n- Not to leave behind too many Go routines in the system.\r\n## Current Behavior\r\n<!--- If describing a bug, tell us what happens instead of the expected behavior -->\r\n<!--- If suggesting a change/improvement, explain the difference from current behavior -->\r\n- Everytime the test is executed the test hangs.\r\n- Test is ran at Concurrency of 1 (just one) . \r\n- The test hangs and if its stopped and re run the number of running Go routines in the system increases by close to 1000. \r\n- After several failed attempts of the tests the number of go routines in the system has increased to 73798  ! \r\n## Possible Solution\r\n<!--- Not obligatory, but suggest a fix/reason for the bug, -->\r\n<!--- or ideas how to implement the addition or change -->\r\n- Need to figure the reason for failure and such a big bump in number of go routines alive in the system.\r\n- Not sure about the origin of the problem.\r\n## Steps to Reproduce (for bugs)\r\n<!--- Provide a link to a live example, or an unambiguous set of steps to -->\r\n<!--- reproduce this bug. Include code to reproduce, if relevant -->\r\n1. Set up 8 node distributed Minio.\r\n2. Run `exec-concurrent TestPutObjectReadAt` test. [Click here](https://github.com/minio/perftest/tree/master/exec-concurrent) for more info on exec-concurrent tests.\r\n\r\n## Context\r\n<!--- How has this issue affected you? What are you trying to accomplish? -->\r\n<!--- Providing context helps us come up with a solution that is most useful in the real world -->\r\n- exec-concurrent tests without chaos test running.\r\n## Your Environment\r\n<!--- Include as many relevant details about the environment you experienced the bug in -->\r\n* Version used: Master at 05dc52a206a47abd71aa1d81e5eb3053285f034a .\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/minio/minio/issues/comments/261709959","html_url":"https://github.com/minio/minio/issues/3274#issuecomment-261709959","issue_url":"https://api.github.com/repos/minio/minio/issues/3274","id":261709959,"user":{"login":"krisis","id":522593,"avatar_url":"https://avatars.githubusercontent.com/u/522593?v=3","gravatar_id":"","url":"https://api.github.com/users/krisis","html_url":"https://github.com/krisis","followers_url":"https://api.github.com/users/krisis/followers","following_url":"https://api.github.com/users/krisis/following{/other_user}","gists_url":"https://api.github.com/users/krisis/gists{/gist_id}","starred_url":"https://api.github.com/users/krisis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krisis/subscriptions","organizations_url":"https://api.github.com/users/krisis/orgs","repos_url":"https://api.github.com/users/krisis/repos","events_url":"https://api.github.com/users/krisis/events{/privacy}","received_events_url":"https://api.github.com/users/krisis/received_events","type":"User","site_admin":false},"created_at":"2016-11-19T12:00:42Z","updated_at":"2016-11-19T12:00:42Z","body":"I am unable to recreate the issue with ``4a926d1`` on master branch using the steps to reproduce provided in the issue description."}}
{"actor":{"display_login":"krisis"}}