<p>For over a year I have my static website deployed in a Kubernetes cluster having only one node (basically respecting <strong>Scenario 1</strong> below).</p>

<p>These days I added another node to my cluster and everything works correctly, except one weird detail which blocked me publishing my website. </p>

<p>To exemplify, let's say I'm in <code>pod1</code>'s shell running on node <code>node1</code>.</p>

<h3>Scenario 1. If my website is running in a pod on the same <code>node1</code>, then</h3>

<pre class="lang-sh prettyprint-override"><code>// pod1

$ wget &lt;static-website-pod-IP&gt;:PORT/some-file.html
</code></pre>

<p>successfully fetches <code>some-file.html</code>.</p>

<h3>Scenario 2. In contrast, if my website is running in a pod on <code>node2</code>, then the same command</h3>

<pre class="lang-sh prettyprint-override"><code>// pod1

$ wget &lt;static-website-pod-IP&gt;:PORT/some-file.html
</code></pre>

<ul>
<li><p>successfully fetches <code>some-file.html</code> if the file has &lt; ~1.4KB (empirically determined).</p></li>
<li><p>blocks forever if <code>some-file.html</code> > ~ 1.4KB.</p></li>
</ul>

<p>For the static website I tried multiple <code>nginx</code> versions and configs and also a python dummy server.</p>

<p>I also tested with Kubernetes latest v1.18 and latest v1.16.</p>

<p>Any idea why I just can't download larger files from pods running on a different node?</p>

svecax
<p>I managed to solve the issue starting from @anton-matsiuk 's hint.</p>

<p>The MTU used by the used network interface is 1450:</p>

<pre><code>$ ip link show

...
3: eth1: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1450 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 86:00:00:53:4d:50 brd ff:ff:ff:ff:ff:ff
...
</code></pre>

<p>So the solution was to modify calico's <code>ConfigMap</code>'s default param <code>veth_mtu: "1440"</code> to <code>veth_mtu: "1430"</code>, according to <a href="https://docs.projectcalico.org/networking/mtu#determine-mtu-size" rel="nofollow noreferrer">this table</a>.</p>

<p>Thank you, @anton-matsiuk.</p>
