<p>I am failing to deploy postgres (single node, official image) on kubernetes and allow services to access postgres via ClusterIP service. </p>

<p>The config is rather simple - Namespace, Deployment, Service:</p>

<pre><code>---
apiVersion: v1
kind: Namespace
metadata:
  name: database
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: database
  name: postgres
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:11.1
          imagePullPolicy: "IfNotPresent"
          ports:
            - containerPort: 5432
---
apiVersion: v1
kind: Service
metadata:
  name: pg
  namespace: database
  labels:
    app: postgres
spec:
  selector:
    app: postgres
  ports:
  - protocol: TCP
    name: postgres
    port: 5432
    targetPort: 5432
</code></pre>

<p>To test is executed a "/bin/bash" into the pod and ran a simple psql command to test the connection. All works well locally:</p>

<pre><code>kubectl --kubeconfig $k8sconf -n database exec -it $(kubectl --kubeconfig $k8sconf -n database get pods -o jsonpath='{.items[*].metadata.name}') -- psql -U admin postgresdb -c "\t"
Tuples only is on.
</code></pre>

<p>But as soon as I try to access postgres via service, the command fails:</p>

<pre><code>kubectl --kubeconfig $k8sconf -n database exec -it $(kubectl --kubeconfig $k8sconf -n database get pods -o jsonpath='{.items[*].metadata.name}') -- psql -h pg -U admin postgresdb -c "\t"
psql: could not connect to server: Connection timed out
    Is the server running on host "pg" (10.245.102.15) and accepting
    TCP/IP connections on port 5432?
</code></pre>

<p>This is tested on a DigitalOcean a single node cluster (1.12.3).</p>

<p>Postgres listened on <code>*</code> on the correct port, <code>pg_hba.conf</code> looks by default like this:</p>

<pre><code>...
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     trust
host    replication     all             127.0.0.1/32            trust
host    replication     all             ::1/128                 trust
host all all all md5
</code></pre>

<p>To reproduce see <a href="https://gist.githubusercontent.com/sontags/c364751e7f0d8ba1a02a9805efc68db6/raw/01b1808348541d743d6a861402cfba224bee8971/database.yaml" rel="nofollow noreferrer">this gist</a></p>

<p>Execute via (please use a fresh cluster and read thru):</p>

<pre><code>export k8sconf=/path/to/your/k8s/confic/file
kubectl --kubeconfig $k8sconf apply -f https://gist.githubusercontent.com/sontags/c364751e7f0d8ba1a02a9805efc68db6/raw/01b1808348541d743d6a861402cfba224bee8971/database.yaml
kubectl --kubeconfig $k8sconf -n database exec -it $(kubectl --kubeconfig $k8sconf -n database get pods -o jsonpath='{.items[*].metadata.name}') -- /bin/bash /reproducer/runtest.sh
</code></pre>

<p>Any hint why the service does not allow to connect or other tests to perform?</p>

<p>Hard to tell without access to your cluster. This works fine on my AWS cluster. Some things to look at:</p>

<ul>
<li>Is the kube-proxy running on all nodes?</li>
<li>Is your network overlay/CNI running on all nodes?</li>
<li>Does this happen with the pg pod only? what about other pods?</li>
<li>DNS seems to be fine since <code>pg</code> is being resolved to <code>10.245.102.15</code></li>
<li>Are your nodes allowing <a href="https://linuxconfig.org/how-to-turn-on-off-ip-forwarding-in-linux" rel="nofollow noreferrer">IP forwarding</a> from the Linux side?</li>
<li>Are your Digital Ocean firewall rules allowing traffic from any source on port <code>5432</code>?  Note that the PodCidr and K8s Service IP range is different the hostCidr (of your droplets).</li>
</ul>
