<p>I will use a very specific way to explain my problem, but I think this is better to be specific than explain in an abstract way...</p>

<p>Say, there is a MongoDB replica set outside of a Kubernetes cluster but in a network. The ip addresses of all members of the replica set were resolved by /etc/hosts in app servers and db servers.</p>

<p>In an experiment/transition phase, I need to access those mongo db servers from kubernetes pods.
However, kubernetes doesn't seem to allow adding custom entries to /etc/hosts in pods/containers.</p>

<p>The MongoDB replica sets are already working with large data set, creating a new replica set in the cluster is not an option.</p>

<p>Because I use GKE, changing any of resources in kube-dns namespace should be avoided I suppose. Configuring or replace kube-dns to be suitable for my need are last thing to try.</p>

<p>Is there a way to resolve ip address of custom hostnames in a Kubernetes cluster?</p>

<p>It is just an idea, but if kube2sky can read some entries of configmap and use them as dns records, it colud be great.
e.g. <code>repl1.mongo.local: 192.168.10.100</code>.</p>

<p>EDIT: I referenced this question from <a href="https://github.com/kubernetes/kubernetes/issues/12337" rel="noreferrer">https://github.com/kubernetes/kubernetes/issues/12337</a></p>

<p>There are 2 possible solutions for this problem now:</p>
<ol>
<li>Pod-wise (Adding the changes to every pod needed to resolve these domains)</li>
<li>cluster-wise (Adding the changes to a central place which all pods have access to, Which is in our case is the <code>DNS</code>)</li>
</ol>
<h3>Let's begin with the pod-wise solution:</h3>
<p>As of Kunbernetes 1.7, It's possible now to add entries to a Pod's <code>/etc/hosts</code> directly using <code>.spec.hostAliases</code></p>
<p>For example: to resolve <code>foo.local</code>, <code>bar.local</code> to <code>127.0.0.1</code> and <code>foo.remote</code>,
<code>bar.remote</code> to <code>10.1.2.3</code>, you can configure HostAliases for a Pod under
<code>.spec.hostAliases</code>:</p>
<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: hostaliases-pod
spec:
  restartPolicy: Never
  hostAliases:
  - ip: &quot;127.0.0.1&quot;
    hostnames:
    - &quot;foo.local&quot;
    - &quot;bar.local&quot;
  - ip: &quot;10.1.2.3&quot;
    hostnames:
    - &quot;foo.remote&quot;
    - &quot;bar.remote&quot;
  containers:
  - name: cat-hosts
    image: busybox
    command:
    - cat
    args:
    - &quot;/etc/hosts&quot;
</code></pre>
<h3>The Cluster-wise solution:</h3>
<p>As of Kubernetes v1.12, <code>CoreDNS</code> is the recommended DNS Server, replacing <code>kube-dns.</code> If your cluster originally used <code>kube-dns</code>, you may still have <code>kube-dns</code> deployed rather than <code>CoreDNS</code>. I'm going to assume that you're using <code>CoreDNS</code> as your K8S DNS.</p>
<p>In CoreDNS it's possible to Add an arbitrary entries inside the cluster domain and that way all pods will resolve this entries directly from the DNS without the need to change each and every <code>/etc/hosts</code> file in every pod.</p>
<p><strong>First:</strong></p>
<p>Let's change the coreos ConfigMap and add required changes:</p>
<pre><code>kubectl edit cm coredns -n kube-system 

apiVersion: v1
kind: ConfigMap
data:
  Corefile: |
    .:53 {
        errors
        health {
          lameduck 5s
        }
        hosts /etc/coredns/customdomains.db example.org {
          fallthrough
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
          pods insecure
          fallthrough in-addr.arpa ip6.arpa
        }
        prometheus :9153
        forward . &quot;/etc/resolv.conf&quot;
        cache 30
        loop
        reload
        loadbalance
    }
  customdomains.db: |
    10.10.1.1 mongo-en-1.example.org
    10.10.1.2 mongo-en-2.example.org
    10.10.1.3 mongo-en-3.example.org
    10.10.1.4 mongo-en-4.example.org
</code></pre>
<p>Basically we added two things:</p>
<ol>
<li><p>The <code>hosts</code> plugin before the <code>kubernetes</code> plugin and used the <code>fallthrough</code> option of the <code>hosts</code> plugin to satisfy our case.</p>
<p>To shed some more lights on the <code>fallthrough</code> option.  Any given backend is usually the final word for its zone - it either returns a result, or it returns NXDOMAIN for the
query. However, occasionally this is not the desired behavior, so some of the plugin support a <code>fallthrough</code> option.
When <code>fallthrough</code> is enabled, instead of returning NXDOMAIN when a record is not found, the plugin will pass the
request down the chain. A backend further down the chain then has the opportunity to handle the request and that backend in our case is <code>kubernetes</code>.</p>
</li>
<li><p>We added a new file to the ConfigMap (<code>customdomains.db</code>) and added our custom domains (<code>mongo-en-*.example.org</code>) in there.</p>
</li>
</ol>
<p>Last thing is to Remember to add the <code>customdomains.db</code> file to the <code>config-volume</code> for the CoreDNS pod template:</p>
<pre><code>kubectl edit -n kube-system deployment coredns
</code></pre>
<pre><code>volumes:
        - name: config-volume
          configMap:
            name: coredns
            items:
            - key: Corefile
              path: Corefile
            - key: customdomains.db
              path: customdomains.db
</code></pre>
<p>and finally to make kubernetes reload CoreDNS (each pod running):</p>
<pre><code>$ kubectl rollout restart -n kube-system deployment/coredns
</code></pre>
