<p>I am trying local Kubernetes(Docker-on-mac), and trying to submit a spark job. The spark job, connects with a PostgreSQL database and do some calculations.</p>

<p>The PostgreSQL is running on my Kube and since I have published it, I can access it from the host via localhost:5432. However, when the spark application is trying to connect to PostgreSQL, it throws </p>

<pre><code>Exception in thread "main" org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
</code></pre>

<p>kubectl cluster-info</p>

<pre><code>Kubernetes master is running at https://kubernetes.docker.internal:6443
KubeDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

</code></pre>

<p>kubectl get service postgresql-published</p>

<p><a href="https://i.stack.imgur.com/xUnpw.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xUnpw.png" alt="enter image description here"></a></p>

<p>kubectl describe service spark-store-1588217023181-driver-svc </p>

<pre><code>Name:              spark-store-1588217023181-driver-svc
Namespace:         default
Labels:            &lt;none&gt;
Annotations:       &lt;none&gt;
Selector:          spark-app-selector=spark-533ecb8556b6439eb938d487cc77c330,spark-role=driver
Type:              ClusterIP
IP:                None
Port:              driver-rpc-port  7078/TCP
TargetPort:        7078/TCP
Endpoints:         &lt;none&gt;
Port:              blockmanager  7079/TCP
TargetPort:        7079/TCP
Endpoints:         &lt;none&gt;
Session Affinity:  None
</code></pre>

<p>How can I make my spark job, have access to PostgreSQL service?</p>

<p><code>localhost</code> is there in <code>EXTERNAL_IP</code> but Kubernetes cluster DNS system(CoreDNS) does not know how to resolve it to an IP address.<code>EXTERNAL_IP</code> is supposed to be resolved by an external DNS server and it's generally meant to be used to connect to Postgres from outside the Kubernetes cluster(i.e from another system or from Kubernetes nodes as well) and not from the inside the cluster(i.e from another pod)</p>

<p>Postgres should be accessible from spark pod via <code>10.106.15.112:5432</code> or <code>postgresql-published:5432</code> because kubernetes cluster DNS system knows how to resolve it.</p>

<p>Test the Postgres connectivity</p>

<pre><code>kubectl run postgresql-postgresql-client --rm --tty -i --restart='Never' --namespace default --image bitnami/postgresql --env="PGPASSWORD=&lt;HERE_YOUR_PASSWORD&gt;" --command -- psql --host &lt;HERE_HOSTNAME=SVC_OR_IP&gt; -U &lt;HERE_USERNAME&gt; 
</code></pre>
