<p>I have a Bare-Metal Kubernetes custom setup (manually setup cluster using Kubernetes the Hard Way). Everything seems to work, but I cannot access services externally.</p>

<p>I can get the list of services when curl:</p>

<pre><code>https://&lt;ip-addr&gt;/api/v1/namespaces/kube-system/services
</code></pre>

<p>However, when I try to proxy (using <code>kubectl proxy</code>, and also by using the <code>&lt;master-ip-address&gt;:&lt;port&gt;</code>):</p>

<pre><code>https://&lt;ip-addr&gt;/api/v1/namespaces/kube-system/services/toned-gecko-grafana:80/proxy/
</code></pre>

<p>I get:</p>

<pre><code>Error: 'dial tcp 10.44.0.16:3000: connect: no route to host'
Trying to reach: 'http://10.44.0.16:3000/'
</code></pre>

<ul>
<li><p><strike>Even if I normally curl <code>http://10.44.0.16:3000/</code> I get the same error. This is the result whether I curl from inside the VM where Kubernetes is installed.</strike> Was able to resolve this, check below.</p></li>
<li><p>I can access my services externally using NodePort.</p></li>
<li><p>I can access my services if I expose them through Nginx-Ingress.</p></li>
<li><p>I am using Weave as CNI, and the logs were normal except a couple of log-lines at the beginning about it not being able to access Namespaces (RBAC error). Though logs were fine after that.</p></li>
<li><p>Using CoreDNS, logs look normal. APIServer and Kubelet logs look normal. Kubernetes-Events look normal, too.</p></li>
<li><p><strong><em>Additional Note</em></strong>: The DNS Service-IP I assigned is <code>10.3.0.10</code>, and the service IP range is: <code>10.3.0.0/24</code>, and POD Network is <code>10.2.0.0/16</code>. I am not sure what <code>10.44.x.x</code> is or where is it coming from.</p>

<ul>
<li>Also, I am using Nginx-Ingress (Helm Chart: <a href="https://github.com/helm/charts/tree/master/stable/nginx-ingress" rel="noreferrer">https://github.com/helm/charts/tree/master/stable/nginx-ingress</a>)</li>
</ul></li>
</ul>

<p>Here is output from one of the services:</p>

<pre><code>{
  "kind": "Service",
  "apiVersion": "v1",
  "metadata": {
    "name": "kubernetes-dashboard",
    "namespace": "kube-system",
    "selfLink": "/api/v1/namespaces/kube-system/services/kubernetes-dashboard",
    "uid": "5c8bb34f-c6a2-11e8-84a7-00163cb4ceeb",
    "resourceVersion": "7054",
    "creationTimestamp": "2018-10-03T00:22:07Z",
    "labels": {
      "addonmanager.kubernetes.io/mode": "Reconcile",
      "k8s-app": "kubernetes-dashboard",
      "kubernetes.io/cluster-service": "true"
    },
    "annotations": {
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{},\"labels\":{\"addonmanager.kubernetes.io/mode\":\"Reconcile\",\"k8s-app\":\"kubernetes-dashboard\",\"kubernetes.io/cluster-service\":\"true\"},\"name\":\"kubernetes-dashboard\",\"namespace\":\"kube-system\"},\"spec\":{\"ports\":[{\"port\":443,\"targetPort\":8443}],\"selector\":{\"k8s-app\":\"kubernetes-dashboard\"}}}\n"
    }
  },
  "spec": {
    "ports": [
      {
        "protocol": "TCP",
        "port": 443,
        "targetPort": 8443,
        "nodePort": 30033
      }
    ],
    "selector": {
      "k8s-app": "kubernetes-dashboard"
    },
    "clusterIP": "10.3.0.30",
    "type": "NodePort",
    "sessionAffinity": "None",
    "externalTrafficPolicy": "Cluster"
  },
  "status": {
    "loadBalancer": {

    }
  }
}
</code></pre>

<p>I am not sure how to debug this, even some pointers to the right direction would help. If anything else is required, please let me know.</p>

<hr>

<p>Output from <code>kubectl get svc</code>:</p>

<pre><code>NAME                   TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
coredns-primary        ClusterIP   10.3.0.10    &lt;none&gt;        53/UDP,53/TCP,9153/TCP   4h51m
kubernetes-dashboard   NodePort    10.3.0.30    &lt;none&gt;        443:30033/TCP            4h51m
</code></pre>

<hr>

<p><strong>EDIT:</strong></p>

<p>Turns out I didn't have <code>kube-dns</code> service running for some reason, despite having CoreDNS running. It was as mentioned here: <a href="https://github.com/kubernetes/kubeadm/issues/1056#issuecomment-413235119" rel="noreferrer">https://github.com/kubernetes/kubeadm/issues/1056#issuecomment-413235119</a></p>

<p>Now I can curl from inside the VM successfully, but the proxy-access still gives me the same error: <code>No route to host</code>. I am not sure why or how would this fix the issue, since I don't see DNS being in play here, but it fixed the issue regardles. Would appreciate any possible explanation on this too.</p>

<p>I encountered the same issue and resolved it by running the commands below:</p>
<pre><code>iptables --flush
iptables -tnat --flush
systemctl stop firewalld
systemctl disable firewalld
systemctl restart docker
</code></pre>
