<p>I have the following configuration to setup the cluster using <a href="https://rancher.com/docs/rancher/v2.x/en/installation/ha/kubernetes-rke/" rel="nofollow noreferrer">Rancher (RKE)</a>.</p>

<p>rancher-config.yml</p>

<pre><code>nodes:
  - address: 192.168.88.204
    internal_address: 172.16.22.12
    user: dockeruser
    role: [controlplane,worker,etcd]
  - address: 192.168.88.203
    internal_address: 172.16.32.37
    user: dockeruser
    role: [controlplane,worker,etcd]
  - address: 192.168.88.202
    internal_address: 172.16.42.73
    user: dockeruser
    role: [controlplane,worker,etcd]

services:
  etcd:
    snapshot: true
    creation: 6h
    retention: 24h 
</code></pre>

<p>According <a href="https://rancher.com/docs/rancher/v2.x/en/installation/requirements/" rel="nofollow noreferrer">Rancher Networking</a>, I already open the following port for all nodes(192.168.88.204, 192.168.88.203, 192.168.88.202) as firewall-services.</p>

<p>node-firewall.xml</p>

<pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt;
&lt;service&gt;
    &lt;port port="2376" protocol="tcp"/&gt;
    &lt;port port="2379" protocol="tcp"/&gt;
    &lt;port port="2380" protocol="tcp"/&gt;
    &lt;port port="8472" protocol="udp"/&gt;
    &lt;port port="9099" protocol="tcp"/&gt;
    &lt;port port="10250" protocol="tcp"/&gt;
    &lt;port port="443" protocol="tcp"/&gt;
    &lt;port port="6443" protocol="tcp"/&gt;
    &lt;port port="8472" protocol="udp"/&gt;
    &lt;port port="6443" protocol="tcp"/&gt;
    &lt;port port="10254" protocol="tcp"/&gt;
    &lt;port port="30000-32767" protocol="tcp"/&gt;
&lt;/service&gt;

-&gt; commmend

firewall-offline-cmd --new-service-from-file=node-firewall.xml --name=node-firewall
firewall-cmd --reload
firewall-cmd --add-service node-firewall
</code></pre>

<p>My RKE is installed on 192.168.88.151. For RKE -> </p>

<pre><code>rancher-firewall.xml

&lt;?xml version="1.0" encoding="utf-8"?&gt;
&lt;service&gt;
    &lt;port port="80" protocol="tcp"/&gt;
    &lt;port port="433" protocol="tcp"/&gt;
    &lt;port port="22" protocol="tcp"/&gt;
    &lt;port port="2376" protocol="tcp"/&gt;
    &lt;port port="6443" protocol="tcp"/&gt;
&lt;/service&gt;
firewall-offline-cmd --new-service-from-file=rancher-firewall.xml --name=rancher-firewall
firewall-cmd --reload
firewall-cmd --add-service rancher-firewall
</code></pre>

<p>So, I run the following commend to up my <code>RKE</code></p>

<pre><code>rke up --config ./rancher-config.yml
</code></pre>

<p>log is</p>

<pre><code>[root@localhost ~]# rke up --config ./rancher-config.yml
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [192.168.88.204]
INFO[0000] [dialer] Setup tunnel for host [192.168.88.203]
INFO[0000] [dialer] Setup tunnel for host [192.168.88.202]
INFO[0001] [network] Deploying port listener containers
INFO[0001] [network] Port listener containers deployed successfully
INFO[0001] [network] Running etcd &lt;-&gt; etcd port checks
INFO[0001] [network] Successfully started [rke-port-checker] container on host [192.168.88.202]
INFO[0001] [network] Successfully started [rke-port-checker] container on host [192.168.88.204]
INFO[0001] [network] Successfully started [rke-port-checker] container on host [192.168.88.203]
FATA[0016] [network] Host [192.168.88.202] is not able to connect to the following ports: 
            [172.16.22.12:2379, 172.16.22.12:2380, 172.16.32.37:2379, 172.16.32.37:2380, 172.16.42.73:2380, 172.16.42.73:2379]. 
            Please check network policies and firewall rules
</code></pre>

<p>My question is how to open the port for the <code>internal_address</code> for all nodes in <code>kubernates</code> cluster?</p>

<p>May be it is lack of my experience. I just share what I found.
<code>internal_address</code> is have to be ip-address of (Gateway) of <code>docker</code>. 
To know the ip-address of docker for each node (192.168.88.204, 192.168.88.203, 192.168.88.202).</p>

<p>Run the commend <code>docker network ls</code>. You might be get following network information.</p>

<pre><code>NETWORK ID          NAME                DRIVER              SCOPE
aa13d08f2676        bridge              bridge              local
02eabe818790        host                host                local
1e5bb430d790        none                null                local
</code></pre>

<p>And run the commend <code>docker network inspect bridge</code> to get ip-addres of <code>bridge</code>.
you will get the following similer info. </p>

<pre><code>[
    {
        "Name": "bridge",
        "Id": "aa13d08f2676e40df5a82521fccc4e402ef6b04f82bcd414cd065a1859b3799d",
        "Created": "2019-01-31T21:32:02.381082005-05:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        ....
        ...
        ..
        .

]
</code></pre>

<p>and configure <code>rancher-config.yml</code> as below and run <code>rke up --config ./rancher-config.yml</code> again</p>

<pre><code>nodes:
  - address: 192.168.88.204
    internal_address: 172.17.0.1
    ...
...
..
..
</code></pre>
