<p>I have a kubernetes single-node setup (see <a href="https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html">https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html</a> )</p>

<p>I have a service and an replication controller creating pods.  Those pods need to connect to the other pods in the same service (Note: this is ultimately so that I can get mongo running w/replica sets (non localhost), but this simple example demonstrates the problem that mongo has).</p>

<p>When I connect from any node to the service, it will be distributed (as expected) to one of the pods.  This will work until it load balances to itself (the container that I am on).  Then it fails to connect.  </p>

<p>Sorry to be verbose, but I am going to attach all my files so that you can see what I'm doing in this little example.</p>

<p>Dockerfile:</p>

<pre><code>FROM ubuntu
MAINTAINER Eric H
RUN apt-get update; apt-get install netcat
EXPOSE 8080
COPY ./entry.sh /
ENTRYPOINT ["/entry.sh"]
</code></pre>

<p>Here is the entry point</p>

<pre><code>#!/bin/bash
# wait for a connection, then tell them who we are 
while : ; do 
    echo "hello, the date=`date`; my host=`hostname`" | nc -l 8080 
    sleep .5
done
</code></pre>

<p>build the dockerfile</p>

<p><code>docker build -t echoserver .</code></p>

<p>tag and upload to my k8s cluster's registry</p>

<pre><code>docker tag -f echoserver:latest 127.0.0.1:5000/echoserver:latest
docker push 127.0.0.1:5000/echoserver:latest
</code></pre>

<p>Here is my Replication Controller</p>

<pre><code>apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    role: echo-server
    app: echo
  name: echo-server-1
spec:
  replicas: 3
  template:
    metadata:
      labels:
        entity: echo-server-1
        role: echo-server
        app: echo
    spec:
      containers:
      - 
        image: 127.0.0.1:5000/echoserver:latest
        name: echo-server-1

        ports:
          - containerPort: 8080
</code></pre>

<p>And finally, here is my Service</p>

<pre><code>kind: Service
metadata:
  labels:
    app: echo
    role: echo-server
    name: echo-server-1
  name: echo-server-1
spec:
  selector:
    entity: echo-server-1
    role: echo-server
  ports:
    - port: 8080
      targetPort: 8080
</code></pre>

<p>Create my service 
<code>kubectl create -f echo.service.yaml</code></p>

<p>Create my rc 
<code>kubectl create -f echo.controller.yaml</code></p>

<p>Get my PODs</p>

<pre><code>kubectl get po
NAME                  READY     STATUS    RESTARTS   AGE
echo-server-1-jp0aj   1/1       Running   0          39m
echo-server-1-shoz0   1/1       Running   0          39m
echo-server-1-y9bv2   1/1       Running   0          39m
</code></pre>

<p>Get the service IP</p>

<pre><code>kubectl get svc
NAME            CLUSTER_IP   EXTERNAL_IP   PORT(S)    SELECTOR                                AGE
echo-server-1   10.3.0.246   &lt;none&gt;        8080/TCP   entity=echo-server-1,role=echo-server   39m
</code></pre>

<p>Exec into one of the pods 
<code>kubectl exec -t -i  echo-server-1-jp0aj /bin/bash</code></p>

<p>Now connect to the service multiple times... It will give me the app-message for all pods except for when it gets to itself, whereupon it hangs.</p>

<pre><code>root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
hello, the date=Mon Jan 11 22:02:38 UTC 2016; my host=echo-server-1-y9bv2
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
^C
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
hello, the date=Mon Jan 11 22:02:43 UTC 2016; my host=echo-server-1-shoz0
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
^C
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
hello, the date=Mon Jan 11 22:31:19 UTC 2016; my host=echo-server-1-y9bv2
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
hello, the date=Mon Jan 11 22:31:23 UTC 2016; my host=echo-server-1-shoz0
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
hello, the date=Mon Jan 11 22:31:26 UTC 2016; my host=echo-server-1-y9bv2
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
hello, the date=Mon Jan 11 22:31:27 UTC 2016; my host=echo-server-1-shoz0
root@echo-server-1-jp0aj:/# nc 10.3.0.246 8080
</code></pre>

<p>How can I configure things so that all members of a service can connect to all other members, including itself?</p>

<p>Thanks to all those who helped on GitHub.<br>
The workaround turned out to be as follows: </p>

<blockquote>
  <p>tanen01 commented on Feb 4 Seeing the same problem here on k8s v1.1.7
  stable</p>
  
  <p>Issue occurs with:</p>
</blockquote>

<pre><code>kube-proxy --proxy-mode=iptables 
</code></pre>

<blockquote>
  <p>Once I changed it to:</p>
</blockquote>

<pre><code>           --proxy-mode=userspace 
</code></pre>

<blockquote>
  <p>(also the default), then it works again.</p>
</blockquote>

<p>So, if you are experiencing this, please try turning off <code>--proxy-mode</code> when you start <code>kube-proxy</code>.</p>
