<p>I create a one-replica zookeeper + kafka cluster with the official kafka chart from the official incubator repo:</p>

<pre><code>helm install --name mykafka -f kafka.yaml incubator/kafka
</code></pre>

<p>This gives me two pods:</p>

<pre><code>kubectl get pods
NAME                  READY     STATUS
mykafka-kafka-0       1/1       Running
mykafka-zookeeper-0   1/1       Running
</code></pre>

<p>And four services (in addition to the default kubernetes service)</p>

<pre><code>kubectl get service
NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)
kubernetes                   ClusterIP   10.96.0.1       &lt;none&gt;        443/TCP
mykafka-kafka                ClusterIP   10.108.143.59   &lt;none&gt;        9092/TCP
mykafka-kafka-headless       ClusterIP   None            &lt;none&gt;        9092/TCP
mykafka-zookeeper            ClusterIP   10.109.43.48    &lt;none&gt;        2181/TCP
mykafka-zookeeper-headless   ClusterIP   None            &lt;none&gt;        2888/TCP,3888/TCP
</code></pre>

<p>If I shell into the zookeeper pod:</p>

<pre><code>&gt; kubectl exec -it mykafka-zookeeper-0 -- /bin/bash
</code></pre>

<p>I use the <code>curl</code> tool to test TCP connectivity. I expect a communications error as the server isn't using HTTP, but if curl can't even connect and I have to ctrl-C out, then the TCP connection isn't working.</p>

<p>I can access the local pod through <code>curl localhost:2181</code>:</p>

<pre><code>root@mykafka-zookeeper-0:/# curl localhost:2181
curl: (52) Empty reply from server
</code></pre>

<p>I can access other pod through <code>curl mykafka-kafka:9092</code>:</p>

<pre><code>root@mykafka-zookeeper-0:/# curl mykafka-kafka:9092
curl: (56) Recv failure: Connection reset by peer
</code></pre>

<p>But I can't access <code>mykafka-zookeeper:2181</code>. That name resolves to the cluster IP, but the attempt to TCP connect hangs until I ctrl-C:</p>

<pre><code>root@mykafka-zookeeper-0:/# curl -v mykafka-zookeeper:2181
* Rebuilt URL to: mykafka-zookeeper:2181/
*   Trying 10.109.43.48...
^C
</code></pre>

<p>Similarly, I can shell into the kafka pod:</p>

<pre><code>&gt; kubectl exec -it mykafka-kafka-0 -- /bin/bash
</code></pre>

<p>Connecting to the Zookeeper pod by the service name works fine:</p>

<pre><code>root@mykafka-kafka-0:/# curl mykafka-zookeeper:2181
curl: (52) Empty reply from server
</code></pre>

<p>Connecting to localhost kafka works fine:</p>

<pre><code>root@mykafka-kafka-0:/# curl localhost:9092
curl: (56) Recv failure: Connection reset by peer
</code></pre>

<p>But connecting to the Kafka pod by the service name doesn't work and I must ctrl-C the curl attempt:</p>

<pre><code>curl -v mykafka-kafka:9092
* Rebuilt URL to: mykafka-kafka:9092/
* Hostname was NOT found in DNS cache
*   Trying 10.108.143.59...
^C
</code></pre>

<p>Can anyone explain why using I can only connect to a Kubernetes service from outside the service and not from within the service?</p>

<p>I believe what you're experiencing can be resolved by looking at how your kubelet is set up to run. There is a setting you can toggle when starting up the kubelet called <code>--hairpin-mode</code>. By default this setting is set to the string <code>promiscuous</code>, where a pod can't connect to its own service, but you can change it to be <code>hairpin-veth</code>, which would allow a pod to connect to its own service.</p>

<p>There are a few issues on the topic, but this seems to be referenced the most:
<a href="https://github.com/kubernetes/kubernetes/issues/45790" rel="nofollow noreferrer">https://github.com/kubernetes/kubernetes/issues/45790</a></p>
