title: Kubernetes Pod to run with OpenVPN client sidecar and have functional DNS through the tunnel and in cluster
body: <p>I attempt to build a Pod that runs a service that requires:</p>
  <ol>
  <li>cluster-internal services to be resolved and accessed by their FQDN (<code>*.cluster.local</code>),</li>
  <li>while also have an active OpenVPN connection to a remote cluster and have services from this remote cluster to be resolved and accessed by their FQDN (<code>*.cluster.remote</code>).</li>
  </ol>
  <p>The service container within the Pod without an OpenVPN sidecar can access all services provided an FQDN using the <code>*.cluster.local</code> namespace. Here is the <code>/etc/resolv.conf</code> in this case:</p>
  <pre><code>nameserver 169.254.25.10
  search default.cluster.local svc.cluster.local cluster.local
  options ndots:5
  </code></pre>
  <h4>When OpenVPN sidecar manages <code>resolv.conf</code></h4>
  <p>The OpenVPN sidecar is started in the following way:</p>
  <pre><code>      containers:
          {{- if .Values.vpn.enabled }}
          - name: vpn
            image: &quot;ghcr.io/wfg/openvpn-client&quot;
            imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
            volumeMounts:
              - name: vpn-working-directory
                mountPath: /data/vpn
            env:
              - name: KILL_SWITCH
                value: &quot;off&quot;
              - name: VPN_CONFIG_FILE
                value: connection.conf
            securityContext:
              privileged: true
              capabilities:
                add:
                  - &quot;NET_ADMIN&quot;
            resources:
              limits:
                cpu: 100m
                memory: 80Mi
              requests:
                cpu: 25m
                memory: 20Mi
          {{- end }}
  </code></pre>
  <p><em>and</em> the OpenVPN client configuration contains the following lines:</p>
  <pre><code>        script-security 2
          up /etc/openvpn/up.sh
          down /etc/openvpn/down.sh
  </code></pre>
  <p>Then OpenVPN client will overwrite <code>resolv.conf</code> so that it contains the following:</p>
  <pre><code>nameserver 192.168.255.1
  options ndots:5
  </code></pre>
  <p>In this case, any service in <code>*.cluster.remote</code> is resolved, but no services from <code>*.cluster.local</code>. This is expected.</p>
  <h4>When OpenVPN sidecar does not manage <code>resolv.conf</code>, but <code>spec.dnsConfig</code> is provided</h4>
  <p>Remove the following lines from the OpenVPN client configuration:</p>
  <pre><code>        script-security 2
          up /etc/openvpn/up.sh
          down /etc/openvpn/down.sh
  </code></pre>
  <p>The <code>spec.dnsConfig</code> is provided as:</p>
  <pre><code>
        dnsConfig:
          nameservers:
            - 192.168.255.1
          searches:
            - cluster.remote
  </code></pre>
  <p>Then, <code>resolv.conf</code> will be the following:</p>
  <pre><code>nameserver 192.168.255.1
  nameserver 169.254.25.10
  search default.cluster.local svc.cluster.local cluster.local  cluster.remote
  options ndots:5
  </code></pre>
  <p>This would work for <code>*.cluster.remote</code>, but not for anything <code>*.cluster.local</code>, because the second nameserver is tried as long as the first times out. I noticed that some folk would get around this limitation by setting up namespace rotation and timeout for 1 second, but this behavior looks very hectic to me, I would not consider this, not even as a workaround. Or maybe I'm missing something. <strong>My first question would be: Could rotation and timeout work in this case?</strong></p>
  <p>My second question would be: is there any way to make <code>*.cluster.local</code> and <code>*.cluster.remote</code> DNS resolves work reliably from the service container inside the Pod <em>and</em> without using something like <code>dnsmasq</code>?</p>
  <p>My third question would be: if <code>dnsmasq</code> is required, how can I configure it, provided, and overwrite <code>resolv.conf</code> by also making sure that the Kubernetes-provided nameserver can be anything (<code>169.254.25.10</code> in this case).</p>
  <p>Best,
  Zoltán</p>