{"action":"created","issue":{"url":"https://api.github.com/repos/docker/docker/issues/24906","repository_url":"https://api.github.com/repos/docker/docker","labels_url":"https://api.github.com/repos/docker/docker/issues/24906/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/24906/comments","events_url":"https://api.github.com/repos/docker/docker/issues/24906/events","html_url":"https://github.com/docker/docker/issues/24906","id":166899277,"number":24906,"title":"1.12.0-rc4: Service can't talk to another service (swarm mode)","user":{"login":"jfbibeau","id":5847314,"avatar_url":"https://avatars.githubusercontent.com/u/5847314?v=3","gravatar_id":"","url":"https://api.github.com/users/jfbibeau","html_url":"https://github.com/jfbibeau","followers_url":"https://api.github.com/users/jfbibeau/followers","following_url":"https://api.github.com/users/jfbibeau/following{/other_user}","gists_url":"https://api.github.com/users/jfbibeau/gists{/gist_id}","starred_url":"https://api.github.com/users/jfbibeau/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfbibeau/subscriptions","organizations_url":"https://api.github.com/users/jfbibeau/orgs","repos_url":"https://api.github.com/users/jfbibeau/repos","events_url":"https://api.github.com/users/jfbibeau/events{/privacy}","received_events_url":"https://api.github.com/users/jfbibeau/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/docker/docker/labels/area/swarm","name":"area/swarm","color":"fbca04"},{"url":"https://api.github.com/repos/docker/docker/labels/group/networking","name":"group/networking","color":"F6FBFF"},{"url":"https://api.github.com/repos/docker/docker/labels/version/1.12","name":"version/1.12","color":"ededed"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":19,"created_at":"2016-07-21T19:31:38Z","updated_at":"2016-07-21T21:24:53Z","closed_at":null,"body":"**Output of `docker version`:**\r\n\r\n```\r\nClient:\r\n Version:      1.12.0-rc4\r\n API version:  1.24\r\n Go version:   go1.6.2\r\n Git commit:   e4a0dbc\r\n Built:        \r\n OS/Arch:      linux/amd64\r\n\r\nServer:\r\n Version:      1.12.0-rc4\r\n API version:  1.24\r\n Go version:   go1.6.2\r\n Git commit:   e4a0dbc\r\n Built:        \r\n OS/Arch:      linux/amd64\r\n```\r\n\r\n\r\n**Output of `docker info`:**\r\n\r\n```\r\nContainers: 1\r\n Running: 1\r\n Paused: 0\r\n Stopped: 0\r\nImages: 5\r\nServer Version: 1.12.0-rc4\r\nStorage Driver: devicemapper\r\n Pool Name: docker-253:1-176165418-pool\r\n Pool Blocksize: 65.54 kB\r\n Base Device Size: 10.74 GB\r\n Backing Filesystem: xfs\r\n Data file: /dev/loop0\r\n Metadata file: /dev/loop1\r\n Data Space Used: 987.5 MB\r\n Data Space Total: 107.4 GB\r\n Data Space Available: 40.39 GB\r\n Metadata Space Used: 1.88 MB\r\n Metadata Space Total: 2.147 GB\r\n Metadata Space Available: 2.146 GB\r\n Thin Pool Minimum Free Space: 10.74 GB\r\n Udev Sync Supported: true\r\n Deferred Removal Enabled: false\r\n Deferred Deletion Enabled: false\r\n Deferred Deleted Device Count: 0\r\n Data loop file: /var/lib/docker/devicemapper/devicemapper/data\r\n WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.\r\n Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata\r\n Library Version: 1.02.107-RHEL7 (2015-10-14)\r\nLogging Driver: json-file\r\nCgroup Driver: cgroupfs\r\nPlugins:\r\n Volume: local\r\n Network: overlay bridge null host\r\nSwarm: active\r\n NodeID: bhhed9vizh7mjdxve52lup55v\r\n IsManager: Yes\r\n Managers: 1\r\n Nodes: 3\r\n CACertHash: sha256:bbc6df07cdf4131d1cf5153b2edc95244c4f7966fa59cb7b6ea11515731d15d7\r\nRuntimes: runc\r\nDefault Runtime: runc\r\nSecurity Options: seccomp\r\nKernel Version: 3.10.0-327.el7.x86_64\r\nOperating System: Red Hat Enterprise Linux Server 7.2 (Maipo)\r\nOSType: linux\r\nArchitecture: x86_64\r\nCPUs: 1\r\nTotal Memory: 5.671 GiB\r\nName: jf-a.localdomain\r\nID: FWPJ:7UNB:OLBN:23Y6:JFNN:ARRB:7MW6:RWIC:RG3P:I36U:RU47:3KE6\r\nDocker Root Dir: /var/lib/docker\r\nDebug Mode (client): false\r\nDebug Mode (server): true\r\n File Descriptors: 45\r\n Goroutines: 139\r\n System Time: 2016-07-21T15:19:10.688153846-04:00\r\n EventsListeners: 1\r\nRegistry: https://index.docker.io/v1/\r\nInsecure Registries:\r\n 127.0.0.0/8\r\n\r\n```\r\n\r\n**Additional environment details (AWS, VirtualBox, physical, etc.):**\r\nVMs on OpenStack\r\n\r\n\r\n**Steps to reproduce the issue:**\r\n1. Spin up 3 node swarm mode cluster, including specifying the --listen-addr param\r\n2. Create 1 instance of tomcat: `docker service create --replicas 1 --publish 8080:8080 --name first_tomcat tomcat:8.0.36-jre8-alpine`\r\n3. Create 1 instance of tomcat (publish a different port): `docker service create --replicas 1 --publish 8081:8080 --name second_tomcat tomcat:8.0.36-jre8-alpine`\r\n4. `docker exec -it <container id of first_tomcat> sh`\r\n5. `wget -O - second_tomcat:8080` (using wget as alpine doesn't come with curl, sorry haha)\r\n\r\n**Describe the results you received:**\r\nTry using service VIP:\r\n```\r\n/usr/local/tomcat # wget -O - second_tomcat:8080\r\nConnecting to second_tomcat:8080 (10.255.0.7:8080)\r\n<timeout>\r\n```\r\n\r\nTry using fixed IP:\r\n```\r\n/usr/local/tomcat # wget -O - tasks.second_tomcat:8080\r\nConnecting to tasks.second_tomcat:8080 (10.255.0.8:8080)\r\n<timeout>\r\n```\r\n\r\n**Describe the results you expected:**\r\nShould be able to reach the other service's instance!\r\n\r\n**Additional information you deem important (e.g. issue happens only occasionally):**\r\nAlways happens. Happens with both an in-house Tomcat container and the public Tomcat container.\r\n\r\nDocker inspect of the second_tomcat container:\r\n\r\n```\r\n# docker inspect 2f539696a2e0\r\n[\r\n    {\r\n        \"Id\": \"2f539696a2e0a228558cd1b25286d6637c88c6f124b3bd72d2f1b8737574b878\",\r\n        \"Created\": \"2016-07-21T19:15:30.727331171Z\",\r\n        \"Path\": \"catalina.sh\",\r\n        \"Args\": [\r\n            \"run\"\r\n        ],\r\n        \"State\": {\r\n            \"Status\": \"running\",\r\n            \"Running\": true,\r\n            \"Paused\": false,\r\n            \"Restarting\": false,\r\n            \"OOMKilled\": false,\r\n            \"Dead\": false,\r\n            \"Pid\": 17356,\r\n            \"ExitCode\": 0,\r\n            \"Error\": \"\",\r\n            \"StartedAt\": \"2016-07-21T19:15:31.380976163Z\",\r\n            \"FinishedAt\": \"0001-01-01T00:00:00Z\"\r\n        },\r\n        \"Image\": \"sha256:b0ca4b09f74556a997ab35150b7995cd722a3d5aa98255404684341b8a5cd6a9\",\r\n        \"ResolvConfPath\": \"/var/lib/docker/containers/2f539696a2e0a228558cd1b25286d6637c88c6f124b3bd72d2f1b8737574b878/resolv.conf\",\r\n        \"HostnamePath\": \"/var/lib/docker/containers/2f539696a2e0a228558cd1b25286d6637c88c6f124b3bd72d2f1b8737574b878/hostname\",\r\n        \"HostsPath\": \"/var/lib/docker/containers/2f539696a2e0a228558cd1b25286d6637c88c6f124b3bd72d2f1b8737574b878/hosts\",\r\n        \"LogPath\": \"/var/lib/docker/containers/2f539696a2e0a228558cd1b25286d6637c88c6f124b3bd72d2f1b8737574b878/2f539696a2e0a228558cd1b25286d6637c88c6f124b3bd72d2f1b8737574b878-json.log\",\r\n        \"Name\": \"/second_tomcat.1.b5u1k8vbr9zs7rcouuscht0d6\",\r\n        \"RestartCount\": 0,\r\n        \"Driver\": \"devicemapper\",\r\n        \"MountLabel\": \"\",\r\n        \"ProcessLabel\": \"\",\r\n        \"AppArmorProfile\": \"\",\r\n        \"ExecIDs\": [\r\n            \"810f39444f184440e75361a2739178539a7d1c40410dbd1a2451dbad82660e4e\"\r\n        ],\r\n        \"HostConfig\": {\r\n            \"Binds\": null,\r\n            \"ContainerIDFile\": \"\",\r\n            \"LogConfig\": {\r\n                \"Type\": \"json-file\",\r\n                \"Config\": {}\r\n            },\r\n            \"NetworkMode\": \"default\",\r\n            \"PortBindings\": null,\r\n            \"RestartPolicy\": {\r\n                \"Name\": \"\",\r\n                \"MaximumRetryCount\": 0\r\n            },\r\n            \"AutoRemove\": false,\r\n            \"VolumeDriver\": \"\",\r\n            \"VolumesFrom\": null,\r\n            \"CapAdd\": null,\r\n            \"CapDrop\": null,\r\n            \"Dns\": null,\r\n            \"DnsOptions\": null,\r\n            \"DnsSearch\": null,\r\n            \"ExtraHosts\": null,\r\n            \"GroupAdd\": null,\r\n            \"IpcMode\": \"\",\r\n            \"Cgroup\": \"\",\r\n            \"Links\": null,\r\n            \"OomScoreAdj\": 0,\r\n            \"PidMode\": \"\",\r\n            \"Privileged\": false,\r\n            \"PublishAllPorts\": false,\r\n            \"ReadonlyRootfs\": false,\r\n            \"SecurityOpt\": null,\r\n            \"UTSMode\": \"\",\r\n            \"UsernsMode\": \"\",\r\n            \"ShmSize\": 67108864,\r\n            \"Runtime\": \"runc\",\r\n            \"ConsoleSize\": [\r\n                0,\r\n                0\r\n            ],\r\n            \"Isolation\": \"\",\r\n            \"CpuShares\": 0,\r\n            \"Memory\": 0,\r\n            \"CgroupParent\": \"\",\r\n            \"BlkioWeight\": 0,\r\n            \"BlkioWeightDevice\": null,\r\n            \"BlkioDeviceReadBps\": null,\r\n            \"BlkioDeviceWriteBps\": null,\r\n            \"BlkioDeviceReadIOps\": null,\r\n            \"BlkioDeviceWriteIOps\": null,\r\n            \"CpuPeriod\": 0,\r\n            \"CpuQuota\": 0,\r\n            \"CpusetCpus\": \"\",\r\n            \"CpusetMems\": \"\",\r\n            \"Devices\": null,\r\n            \"DiskQuota\": 0,\r\n            \"KernelMemory\": 0,\r\n            \"MemoryReservation\": 0,\r\n            \"MemorySwap\": 0,\r\n            \"MemorySwappiness\": -1,\r\n            \"OomKillDisable\": false,\r\n            \"PidsLimit\": 0,\r\n            \"Ulimits\": null,\r\n            \"CpuCount\": 0,\r\n            \"CpuPercent\": 0,\r\n            \"IOMaximumIOps\": 0,\r\n            \"IOMaximumBandwidth\": 0\r\n        },\r\n        \"GraphDriver\": {\r\n            \"Name\": \"devicemapper\",\r\n            \"Data\": {\r\n                \"DeviceId\": \"37\",\r\n                \"DeviceName\": \"docker-253:1-192941280-3a2635d8b7239a777fe02df652422d11fa7bb403fe040871b0997660e6fa284b\",\r\n                \"DeviceSize\": \"10737418240\"\r\n            }\r\n        },\r\n        \"Mounts\": [],\r\n        \"Config\": {\r\n            \"Hostname\": \"2f539696a2e0\",\r\n            \"Domainname\": \"\",\r\n            \"User\": \"\",\r\n            \"AttachStdin\": false,\r\n            \"AttachStdout\": false,\r\n            \"AttachStderr\": false,\r\n            \"ExposedPorts\": {\r\n                \"8080/tcp\": {}\r\n            },\r\n            \"Tty\": false,\r\n            \"OpenStdin\": false,\r\n            \"StdinOnce\": false,\r\n            \"Env\": [\r\n                \"PATH=/usr/local/tomcat/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/jvm/java-1.8-openjdk/jre/bin:/usr/lib/jvm/java-1.8-openjdk/bin\",\r\n                \"LANG=C.UTF-8\",\r\n                \"JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk/jre\",\r\n                \"JAVA_VERSION=8u92\",\r\n                \"JAVA_ALPINE_VERSION=8.92.14-r1\",\r\n                \"CATALINA_HOME=/usr/local/tomcat\",\r\n                \"TOMCAT_NATIVE_LIBDIR=/usr/local/tomcat/native-jni-lib\",\r\n                \"LD_LIBRARY_PATH=/usr/local/tomcat/native-jni-lib\",\r\n                \"TOMCAT_MAJOR=8\",\r\n                \"TOMCAT_VERSION=8.0.36\",\r\n                \"TOMCAT_TGZ_URL=https://www.apache.org/dist/tomcat/tomcat-8/v8.0.36/bin/apache-tomcat-8.0.36.tar.gz\"\r\n            ],\r\n            \"Cmd\": [\r\n                \"catalina.sh\",\r\n                \"run\"\r\n            ],\r\n            \"Image\": \"tomcat:8.0.36-jre8-alpine\",\r\n            \"Volumes\": null,\r\n            \"WorkingDir\": \"/usr/local/tomcat\",\r\n            \"Entrypoint\": null,\r\n            \"OnBuild\": null,\r\n            \"Labels\": {\r\n                \"com.docker.swarm.node.id\": \"70vxyod2eboxcxd8a0wo0ds7n\",\r\n                \"com.docker.swarm.service.id\": \"5d01jg29r59l7d9rs7o0y2u25\",\r\n                \"com.docker.swarm.service.name\": \"second_tomcat\",\r\n                \"com.docker.swarm.task\": \"\",\r\n                \"com.docker.swarm.task.id\": \"b5u1k8vbr9zs7rcouuscht0d6\",\r\n                \"com.docker.swarm.task.name\": \"second_tomcat.1\"\r\n            }\r\n        },\r\n        \"NetworkSettings\": {\r\n            \"Bridge\": \"\",\r\n            \"SandboxID\": \"6e37aebafcb111ca2e1fae9e377e0ff9f60857be123c45851e69719aee285d80\",\r\n            \"HairpinMode\": false,\r\n            \"LinkLocalIPv6Address\": \"\",\r\n            \"LinkLocalIPv6PrefixLen\": 0,\r\n            \"Ports\": {\r\n                \"8080/tcp\": null\r\n            },\r\n            \"SandboxKey\": \"/var/run/docker/netns/6e37aebafcb1\",\r\n            \"SecondaryIPAddresses\": null,\r\n            \"SecondaryIPv6Addresses\": null,\r\n            \"EndpointID\": \"\",\r\n            \"Gateway\": \"\",\r\n            \"GlobalIPv6Address\": \"\",\r\n            \"GlobalIPv6PrefixLen\": 0,\r\n            \"IPAddress\": \"\",\r\n            \"IPPrefixLen\": 0,\r\n            \"IPv6Gateway\": \"\",\r\n            \"MacAddress\": \"\",\r\n            \"Networks\": {\r\n                \"ingress\": {\r\n                    \"IPAMConfig\": {\r\n                        \"IPv4Address\": \"10.255.0.8\"\r\n                    },\r\n                    \"Links\": null,\r\n                    \"Aliases\": [\r\n                        \"2f539696a2e0\"\r\n                    ],\r\n                    \"NetworkID\": \"dzsgwnyjip8yqi4893da420md\",\r\n                    \"EndpointID\": \"0146171a88c911c30d8fc54abf87e51672cb9f4c4ed4146ff2b701d3751c0913\",\r\n                    \"Gateway\": \"\",\r\n                    \"IPAddress\": \"10.255.0.8\",\r\n                    \"IPPrefixLen\": 16,\r\n                    \"IPv6Gateway\": \"\",\r\n                    \"GlobalIPv6Address\": \"\",\r\n                    \"GlobalIPv6PrefixLen\": 0,\r\n                    \"MacAddress\": \"02:42:0a:ff:00:08\"\r\n                }\r\n            }\r\n        }\r\n    }\r\n]\r\n```\r\n\r\nDocker service inspect output:\r\n```\r\n# docker service inspect second_tomcat\r\n[\r\n    {\r\n        \"ID\": \"5d01jg29r59l7d9rs7o0y2u25\",\r\n        \"Version\": {\r\n            \"Index\": 1043\r\n        },\r\n        \"CreatedAt\": \"2016-07-21T19:13:14.020746042Z\",\r\n        \"UpdatedAt\": \"2016-07-21T19:13:14.029082747Z\",\r\n        \"Spec\": {\r\n            \"Name\": \"second_tomcat\",\r\n            \"TaskTemplate\": {\r\n                \"ContainerSpec\": {\r\n                    \"Image\": \"tomcat:8.0.36-jre8-alpine\"\r\n                },\r\n                \"Resources\": {\r\n                    \"Limits\": {},\r\n                    \"Reservations\": {}\r\n                },\r\n                \"RestartPolicy\": {\r\n                    \"Condition\": \"any\",\r\n                    \"MaxAttempts\": 0\r\n                },\r\n                \"Placement\": {}\r\n            },\r\n            \"Mode\": {\r\n                \"Replicated\": {\r\n                    \"Replicas\": 1\r\n                }\r\n            },\r\n            \"UpdateConfig\": {},\r\n            \"EndpointSpec\": {\r\n                \"Mode\": \"vip\",\r\n                \"Ports\": [\r\n                    {\r\n                        \"Protocol\": \"tcp\",\r\n                        \"TargetPort\": 8080,\r\n                        \"PublishedPort\": 8081\r\n                    }\r\n                ]\r\n            }\r\n        },\r\n        \"Endpoint\": {\r\n            \"Spec\": {\r\n                \"Mode\": \"vip\",\r\n                \"Ports\": [\r\n                    {\r\n                        \"Protocol\": \"tcp\",\r\n                        \"TargetPort\": 8080,\r\n                        \"PublishedPort\": 8081\r\n                    }\r\n                ]\r\n            },\r\n            \"Ports\": [\r\n                {\r\n                    \"Protocol\": \"tcp\",\r\n                    \"TargetPort\": 8080,\r\n                    \"PublishedPort\": 8081\r\n                }\r\n            ],\r\n            \"VirtualIPs\": [\r\n                {\r\n                    \"NetworkID\": \"dzsgwnyjip8yqi4893da420md\",\r\n                    \"Addr\": \"10.255.0.7/16\"\r\n                }\r\n            ]\r\n        }\r\n    }\r\n]\r\n```\r\n\r\nAnything else you need or want me to try, let me know!"},"comment":{"url":"https://api.github.com/repos/docker/docker/issues/comments/234388790","html_url":"https://github.com/docker/docker/issues/24906#issuecomment-234388790","issue_url":"https://api.github.com/repos/docker/docker/issues/24906","id":234388790,"user":{"login":"jfbibeau","id":5847314,"avatar_url":"https://avatars.githubusercontent.com/u/5847314?v=3","gravatar_id":"","url":"https://api.github.com/users/jfbibeau","html_url":"https://github.com/jfbibeau","followers_url":"https://api.github.com/users/jfbibeau/followers","following_url":"https://api.github.com/users/jfbibeau/following{/other_user}","gists_url":"https://api.github.com/users/jfbibeau/gists{/gist_id}","starred_url":"https://api.github.com/users/jfbibeau/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfbibeau/subscriptions","organizations_url":"https://api.github.com/users/jfbibeau/orgs","repos_url":"https://api.github.com/users/jfbibeau/repos","events_url":"https://api.github.com/users/jfbibeau/events{/privacy}","received_events_url":"https://api.github.com/users/jfbibeau/received_events","type":"User","site_admin":false},"created_at":"2016-07-21T21:24:53Z","updated_at":"2016-07-21T21:24:53Z","body":"`docker service create` doesn't seem to support that (yet?) :S  Anything else I can try?"}}
{"actor":{"display_login":"jfbibeau"}}