<p>I have implemented a multi master HA kubernetes cluster and wanted to implement the Calico the hardway as described in <a href="https://docs.projectcalico.org/getting-started/kubernetes/hardway/" rel="nofollow noreferrer">here</a>. I was able complete all the steps and my connectivity is not there between the pods and services and pods and other pods in different nodes.</p>
<p>only, difference is I use two different AZs in AWS and I suppose it should not be an issue. I can see pods are getting the IPs and calico network interfaces are creating but still see the connectivity as I explained. Node even doesn't have the public internet access. I did the BGP configuration exactly same in the guide but no luck and I'm not quite sure something to be changed in the BGP configuration when it comes to multi-AZ deployment. I'm not much aware of the Calico BGP configuration.</p>
<p>Unfortunately, calicoctl node diags does not properly run and not providing much more information to move forward.</p>
<p>I'd love here your valuable thoughts and constructive criticism to fix this.</p>

Aruna Fernando
<p>Calico configured in <a href="https://docs.projectcalico.org/v3.7/networking/service-advertisement#about-advertising-kubernetes-services-over-bgp" rel="nofollow noreferrer">BGP mode</a> requires all of the instances to be located in the same subnet to work out of the box.</p>
<p>To use calico with deployments that are split across multiple availability zones you must:</p>
<h3>Disable AWS source / destination check (see here):</h3>
<p><strong>You can do that using AWS CLI:</strong></p>
<pre><code>    aws ec2 modify-instance-attribute --no-source-dest-check --instance-id          $EC2_INSTANCE_ID --region &lt;REGION-WHERE-EC2-INSTANCE-IS-LAUNCHED&gt;
</code></pre>
<p><strong>Or using the AWS console:</strong></p>
<blockquote>
<ol>
<li>Open the Amazon EC2 console at  <a href="https://console.aws.amazon.com/ec2/" rel="nofollow noreferrer">https://console.aws.amazon.com/ec2/</a>.</li>
<li>In the navigation pane, choose  <strong>Instances</strong>.</li>
<li>Select the NAT instance, choose  <strong>Actions</strong>,  <strong>Networking</strong>,  <strong>Change Source/Dest. Check</strong>.</li>
<li>For the NAT instance, verify that this attribute is disabled. Otherwise, choose  <strong>Yes, Disable</strong>.</li>
<li>If the NAT instance has a secondary network interface, choose it from  <strong>Network interfaces</strong>  on the  <strong>Description</strong>  tab and choose
the interface ID to go to the network interfaces page. Choose
<strong>Actions</strong>,  <strong>Change Source/Dest. Check</strong>, disable the setting, and choose  <strong>Save</strong>.*</li>
</ol>
</blockquote>
<h3>Enable IPIP encapsulation and outgoing NAT on your Calico IP pools</h3>
<blockquote>
<p>(<code>IPPool</code>) represents a collection of IP addresses from which Calico
expects endpoint IPs to be assigned. (<a href="https://docs.projectcalico.org/reference/resources/ippool" rel="nofollow noreferrer">see here how to set it up</a>)</p>
</blockquote>
<p>, then all of the Kubernetes instances must be located in the same subnet for Calico to work out of the box.</p>
<p>To enable the “CrossSubnet” IPIP feature, configure your Calico IP pool resources to enable IPIP mode to “CrossSubnet” like in the example below:</p>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: ippool-multi-az
spec:
  cidr: 192.168.0.0/16
  ipipMode: CrossSubnet
EOF
</code></pre>
<p>Example above refers to <a href="https://docs.projectcalico.org/reference/public-cloud/aws" rel="nofollow noreferrer">AWS cloud configuration</a> taken from the Calico documentation. Please note that Calico docs has also information about <a href="https://docs.projectcalico.org/reference/public-cloud/" rel="nofollow noreferrer">GCP, Azure and IBM</a>.</p>
<p>Remark: If you face another problems going &quot;the hard way&quot; you may want to use as a reference another cluster created by following calico guides below:</p>
<ul>
<li><a href="https://docs.projectcalico.org/getting-started/openshift/requirements#network-requirements" rel="nofollow noreferrer">Getting started - openshift</a></li>
<li><a href="https://docs.projectcalico.org/getting-started/kubernetes/self-managed-public-cloud/gce" rel="nofollow noreferrer">Getting started - self managed public cloud -   GCE</a></li>
<li><a href="https://docs.projectcalico.org/getting-started/kubernetes/self-managed-public-cloud/aws" rel="nofollow noreferrer">Getting started - self managed public cloud - AWS</a></li>
</ul>
<p>Lastly, it is worth to check is also this very good document about <a href="https://octetz.com/docs/2020/2020-10-01-calico-routing-modes/" rel="nofollow noreferrer">calico routing modes</a> (it shows also cross subnets ipip mode).</p>
