{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/24443","repository_url":"https://api.github.com/repos/docker/docker","labels_url":"https://api.github.com/repos/docker/docker/issues/24443/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/24443/comments","events_url":"https://api.github.com/repos/docker/docker/issues/24443/events","html_url":"https://github.com/docker/docker/issues/24443","id":164500559,"number":24443,"title":"update of a service stops routing HTTP requests to the new version on Docker Swarm AWS 1.12rc3","user":{"login":"mvanholsteijn","id":576619,"avatar_url":"https://avatars.githubusercontent.com/u/576619?v=3","gravatar_id":"","url":"https://api.github.com/users/mvanholsteijn","html_url":"https://github.com/mvanholsteijn","followers_url":"https://api.github.com/users/mvanholsteijn/followers","following_url":"https://api.github.com/users/mvanholsteijn/following{/other_user}","gists_url":"https://api.github.com/users/mvanholsteijn/gists{/gist_id}","starred_url":"https://api.github.com/users/mvanholsteijn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mvanholsteijn/subscriptions","organizations_url":"https://api.github.com/users/mvanholsteijn/orgs","repos_url":"https://api.github.com/users/mvanholsteijn/repos","events_url":"https://api.github.com/users/mvanholsteijn/events{/privacy}","received_events_url":"https://api.github.com/users/mvanholsteijn/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/docker/docker/labels/version/1.12","name":"version/1.12","color":"ededed"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-07-08T10:14:14Z","updated_at":"2016-07-08T10:47:32Z","closed_at":null,"body":"**Output of `docker version`:**\r\n\r\n```\r\n~ $ docker version\r\nClient:\r\n Version:      1.12.0-rc3\r\n API version:  1.24\r\n Go version:   go1.6.2\r\n Git commit:   91e29e8\r\n Built:        Sat Jul  2 00:09:24 2016\r\n OS/Arch:      linux/amd64\r\n Experimental: true\r\n\r\nServer:\r\n Version:      1.12.0-rc3\r\n API version:  1.24\r\n Go version:   go1.6.2\r\n Git commit:   91e29e8\r\n Built:        Sat Jul  2 00:09:24 2016\r\n OS/Arch:      linux/amd64\r\n Experimental: true\r\n```\r\n\r\n\r\n**Output of `docker info`:**\r\n\r\n```\r\n~ $ docker version\r\nClient:\r\n Version:      1.12.0-rc3\r\n API version:  1.24\r\n Go version:   go1.6.2\r\n Git commit:   91e29e8\r\n Built:        Sat Jul  2 00:09:24 2016\r\n OS/Arch:      linux/amd64\r\n Experimental: true\r\n\r\nServer:\r\n Version:      1.12.0-rc3\r\n API version:  1.24\r\n Go version:   go1.6.2\r\n Git commit:   91e29e8\r\n Built:        Sat Jul  2 00:09:24 2016\r\n OS/Arch:      linux/amd64\r\n Experimental: true\r\n~ $ docker info\r\nContainers: 8\r\n Running: 4\r\n Paused: 0\r\n Stopped: 4\r\nImages: 7\r\nServer Version: 1.12.0-rc3\r\nStorage Driver: aufs\r\n Root Dir: /var/lib/docker/aufs\r\n Backing Filesystem: extfs\r\n Dirs: 52\r\n Dirperm1 Supported: true\r\nLogging Driver: json-file\r\nCgroup Driver: cgroupfs\r\nPlugins:\r\n Volume: local\r\n Network: bridge overlay null host\r\nSwarm: active\r\n NodeID: 1fc6ryd8yz8lr8sydf9y6ga07\r\n IsManager: Yes\r\n Managers: 3\r\n Nodes: 9\r\n CACertHash: sha256:56996486b60c33a5ce82198c70af52c570e10ae03848eea8e6d89755c5678343\r\nRuntimes: runc\r\nDefault Runtime: runc\r\nSecurity Options: seccomp\r\nKernel Version: 4.4.13-moby\r\nOperating System: Alpine Linux v3.4\r\nOSType: linux\r\nArchitecture: x86_64\r\nCPUs: 1\r\nTotal Memory: 993.8 MiB\r\nName: ip-192-168-33-248.eu-west-1.compute.internal\r\nID: AYAF:MDQY:RBBR:MPGR:53L7:LO62:XUD3:U7DA:HKOU:VSHQ:PNGN:K77L\r\nDocker Root Dir: /var/lib/docker\r\nDebug Mode (client): false\r\nDebug Mode (server): true\r\n File Descriptors: 63\r\n Goroutines: 141\r\n System Time: 2016-07-08T09:40:45.002445313Z\r\n EventsListeners: 0\r\nRegistry: https://index.docker.io/v1/\r\nExperimental: true\r\nInsecure Registries:\r\n 127.0.0.0/8```\r\n\r\n**Additional environment details (AWS, VirtualBox, physical, etc.):**\r\n\r\nI am running the CFN template https://docker-for-aws.s3.amazonaws.com/v1.12.0-rc3-beta1/docker_for_aws.json\r\n\r\n**Steps to reproduce the issue:**\r\n1.  docker network create -d overlay paas-monitor\r\n2.  docker service create --name paas-monitor --replicas 5 --network paas-monitor -p :80:1337/tcp  mvanholsteijn/paas-monitor:1.6.2\r\n3. goto http://<your-external-DNS-elb>  (here you will see that the one instance gets all the traffic)\r\n4.  docker service update paas-monitor --image mvanholsteijn/paas-monitor:1.6.3  (Update to add 'connection:close'; to http response)\r\n\r\nThe attached movie shows what the expected result is.\r\n\r\n**Describe the results you received:**\r\n\r\nNo traffic gets routed anymore to the new instances and/or are responses are very very slow.\r\n\r\n**Describe the results you expected:**\r\nnormal network connectivity after service update from the load balancer.\r\n\r\n**Additional information you deem important (e.g. issue happens only occasionally):**\r\n- We needed to add 'Connection:Close' to the response header to get round-robin loadbalancing. Is this by design? \r\n- the AWS ELB says all instances are healthy.\r\n- same problem can be reproduced on own installation of Swarm 1.12 on VirtualBox\r\n\r\n\r\n[expected-result.mov.zip](https://github.com/docker/docker/files/353942/expected-result.mov.zip)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/231330937","html_url":"https://github.com/docker/docker/issues/24443#issuecomment-231330937","issue_url":"https://api.github.com/repos/docker/docker/issues/24443","id":231330937,"user":{"login":"mavenugo","id":5056191,"avatar_url":"https://avatars.githubusercontent.com/u/5056191?v=3","gravatar_id":"","url":"https://api.github.com/users/mavenugo","html_url":"https://github.com/mavenugo","followers_url":"https://api.github.com/users/mavenugo/followers","following_url":"https://api.github.com/users/mavenugo/following{/other_user}","gists_url":"https://api.github.com/users/mavenugo/gists{/gist_id}","starred_url":"https://api.github.com/users/mavenugo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mavenugo/subscriptions","organizations_url":"https://api.github.com/users/mavenugo/orgs","repos_url":"https://api.github.com/users/mavenugo/repos","events_url":"https://api.github.com/users/mavenugo/events{/privacy}","received_events_url":"https://api.github.com/users/mavenugo/received_events","type":"User","site_admin":false},"created_at":"2016-07-08T10:47:32Z","updated_at":"2016-07-08T10:47:32Z","body":"@mvanholsteijn Thanks for reporting the issue.\r\n\r\n> - We needed to add 'Connection:Close' to the response header to get round-robin loadbalancing. Is this by design? \r\n\r\nBased on our earlier investigation, we identified that while accessing your application via the browser, it caches the TCP connection and hence even if you refresh the page, it still maintains the connection. Hence adding an explicit `Connection:Close` force releases the cache and hence the LB works properly if one tries the load-balancing from the same browser instance. But if you use Curl, you will see the LB working properly. Also, if you try this from a different client/browser, you can see that the RR working fine.\r\n\r\n> No traffic gets routed anymore to the new instances and/or are responses are very very slow.\r\n\r\nThis seems like a bug & I will try to reproduce it & identify the root-cause."}}
{"actor":{"display_login":"mavenugo"}}