{"action":"created","issue":{"url":"https://api.github.com/repos/Azure/azure-sdk-for-go/issues/1500","repository_url":"https://api.github.com/repos/Azure/azure-sdk-for-go","labels_url":"https://api.github.com/repos/Azure/azure-sdk-for-go/issues/1500/labels{/name}","comments_url":"https://api.github.com/repos/Azure/azure-sdk-for-go/issues/1500/comments","events_url":"https://api.github.com/repos/Azure/azure-sdk-for-go/issues/1500/events","html_url":"https://github.com/Azure/azure-sdk-for-go/issues/1500","id":311048317,"number":1500,"title":"Unexpected quantity of TCP connections","user":{"login":"meirf","id":2043305,"avatar_url":"https://avatars0.githubusercontent.com/u/2043305?v=4","gravatar_id":"","url":"https://api.github.com/users/meirf","html_url":"https://github.com/meirf","followers_url":"https://api.github.com/users/meirf/followers","following_url":"https://api.github.com/users/meirf/following{/other_user}","gists_url":"https://api.github.com/users/meirf/gists{/gist_id}","starred_url":"https://api.github.com/users/meirf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meirf/subscriptions","organizations_url":"https://api.github.com/users/meirf/orgs","repos_url":"https://api.github.com/users/meirf/repos","events_url":"https://api.github.com/users/meirf/events{/privacy}","received_events_url":"https://api.github.com/users/meirf/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-04T00:55:53Z","updated_at":"2018-04-09T03:15:30Z","closed_at":null,"author_association":"NONE","body":"I ran this code at https://github.com/Azure/azure-sdk-for-go/commit/56332fec5b308fbb6615fa1af6117394cdba186d\r\n\r\nI'm seeing an unexpected number of TCP connections when copying files.\r\n\r\n```\r\nclient, err := storage.NewBasicClientOnSovereignCloud(accountName, accountKey, env)\r\n\r\nclient.HTTPClient.Transport = http.DefaultTransport\r\ntransp, ok := client.HTTPClient.Transport.(*http.Transport)\r\nclient.HTTPClient.Transport = http.DefaultTransport\r\ntransp.MaxIdleConnsPerHost = 200\r\n\r\nrootClient := client.GetContainerReference(\"foo\")\r\nnumRoutines := 200\r\nnumRequestsPerRoutine := 100\r\n\r\nfor i := 0; i < numRoutines; i++ {\r\n\t\twait.Add(1)\r\n\t\tgo func() {\r\n\t\t\tfor j := 0; j < numRequestsPerRoutine; j++ {\r\n\t\t\t\tpath := getRandomPath()\r\n\t\t\t\tsourceBlob := rootClient.GetBlobReference(path)\r\n\t\t\t\tdestBlob := rootClient.GetBlobReference(target + path)\r\n\t\t\t\terr = destBlob.Copy(sourceBlob.GetURL(), nil)\r\n                                time.Sleep(250 * time.Millisecond)\r\n\t\t\t}\r\n\t\t\twait.Done()\r\n\t\t}()\r\n\t}\r\nwait.Wait()\r\ntime.Sleep(5 * time.Minute)\r\n```\r\n\r\nBecause `MaxIdleConnsPerHost` is set to a high number we can reuse a lot more descriptors, but I still see more than what I expected. \r\n\r\n**Expected**\r\nI expected there to be a maximum of 200 connections since I assume each goroutine is executing http requests in serial. \r\n\r\n**Observed**\r\nUntil the end of the execution, the number of TCP connections stays under 200. But towards the end that number consistently jumps to over 400 and sometimes 800 and I start getting `dial tcp: lookup `FOO.blob.core.cloudapi.de on 10.1.2.3:53: dial udp 10.1.2.3:53: socket: too many open files)`\r\n After the copying ends, the connections go from ESTABLISHED to CLOSE presumably because the 90 second idle connection timeout has elapsed. _But the file descriptor is still there even though the connection is closed which I think implies it's not properly being closed._\r\n\r\nHow could there be more than 200 open files if there are only 200 routines?"},"comment":{"url":"https://api.github.com/repos/Azure/azure-sdk-for-go/issues/comments/379617166","html_url":"https://github.com/Azure/azure-sdk-for-go/issues/1500#issuecomment-379617166","issue_url":"https://api.github.com/repos/Azure/azure-sdk-for-go/issues/1500","id":379617166,"user":{"login":"meirf","id":2043305,"avatar_url":"https://avatars0.githubusercontent.com/u/2043305?v=4","gravatar_id":"","url":"https://api.github.com/users/meirf","html_url":"https://github.com/meirf","followers_url":"https://api.github.com/users/meirf/followers","following_url":"https://api.github.com/users/meirf/following{/other_user}","gists_url":"https://api.github.com/users/meirf/gists{/gist_id}","starred_url":"https://api.github.com/users/meirf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meirf/subscriptions","organizations_url":"https://api.github.com/users/meirf/orgs","repos_url":"https://api.github.com/users/meirf/repos","events_url":"https://api.github.com/users/meirf/events{/privacy}","received_events_url":"https://api.github.com/users/meirf/received_events","type":"User","site_admin":false},"created_at":"2018-04-09T03:15:30Z","updated_at":"2018-04-09T03:15:30Z","author_association":"NONE","body":"If I had some certainty around `azure-storage-blob-go` not having this same problem, I would probably switch, but it looks like there is a similar problem there. https://github.com/Azure/azure-storage-blob-go/issues/30\r\n\r\n@jhendrixMSFT \r\nI would be happy with uploading a PR if you could help me narrow down where the problem is. I appreciate any suggestions on this.\r\n\r\nThank you!!"}}
{"actor":{"display_login":"meirf"}}