title: route not working in kubernetes with calico
body: <p>I have</p>
  
  <ul>
  <li>kubernetes v1.6.0 setup by kubeadm v1.6.1</li>
  <li>calico setup by offical <a href="http://docs.projectcalico.org/v2.1/getting-started/kubernetes/installation/hosted/kubeadm/1.6/calico.yaml" rel="nofollow noreferrer">yaml</a></li>
  <li>iptables v1.6.0</li>
  <li>nodes are provided by AliCloud</li>
  </ul>
  
  <p>Problem:</p>
  
  <p>The cni network is not working. Any deployment can only be visited from the node where it is running. I doubt it is related with route table conflict/missing, because I have another cluster on Vultr Cloud working fine, with the same setup steps.</p>
  
  <p>Cluster Info:</p>
  
  <pre><code>root@iZ2ze8ctk2q17u029a8wcoZ:~# kubectl get pods --all-namespaces -o wide
  NAMESPACE     NAME                                              READY     STATUS    RESTARTS   AGE       IP               NODE
  kube-system   calico-etcd-66gf4                                 1/1       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  kube-system   calico-node-4wxsb                                 2/2       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  kube-system   calico-node-6n1g1                                 2/2       Running   0          16h       10.30.248.80     iz2zegw6nmd5t5qxy35lh0z
  kube-system   calico-policy-controller-2561685917-7bdd4         1/1       Running   0          16h       10.30.248.80     iz2zegw6nmd5t5qxy35lh0z
  kube-system   etcd-iz2ze8ctk2q17u029a8wcoz                      1/1       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  kube-system   heapster-bx03l                                    1/1       Running   0          16h       192.168.31.150   iz2zegw6nmd5t5qxy35lh0z
  kube-system   kube-apiserver-iz2ze8ctk2q17u029a8wcoz            1/1       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  kube-system   kube-controller-manager-iz2ze8ctk2q17u029a8wcoz   1/1       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  kube-system   kube-dns-3913472980-kgzln                         3/3       Running   0          16h       192.168.31.149   iz2zegw6nmd5t5qxy35lh0z
  kube-system   kube-proxy-ck83t                                  1/1       Running   0          16h       10.30.248.80     iz2zegw6nmd5t5qxy35lh0z
  kube-system   kube-proxy-lssdn                                  1/1       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  kube-system   kube-scheduler-iz2ze8ctk2q17u029a8wcoz            1/1       Running   0          16h       10.27.219.50     iz2ze8ctk2q17u029a8wcoz
  </code></pre>
  
  <p>I checked each pod's log, cannot find anything wrong.</p>
  
  <p>Master Info:
  internal ip: 10.27.219.50</p>
  
  <pre><code>root@iZ2ze8ctk2q17u029a8wcoZ:~# ifconfig
  
  docker0   Link encap:Ethernet  HWaddr 02:42:56:84:35:19
            inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.255.0
            UP BROADCAST MULTICAST  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:0
            RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
  
  eth0      Link encap:Ethernet  HWaddr 00:16:3e:30:51:ae
            inet addr:10.27.219.50  Bcast:10.27.219.255  Mask:255.255.252.0
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:4400927 errors:0 dropped:0 overruns:0 frame:0
            TX packets:3906530 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:564808928 (564.8 MB)  TX bytes:792611382 (792.6 MB)
  
  eth1      Link encap:Ethernet  HWaddr 00:16:3e:32:07:f8
            inet addr:59.110.32.199  Bcast:59.110.35.255  Mask:255.255.252.0
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:1148756 errors:0 dropped:0 overruns:0 frame:0
            TX packets:688177 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:1570341044 (1.5 GB)  TX bytes:58104611 (58.1 MB)
  
  tunl0     Link encap:IPIP Tunnel  HWaddr
            inet addr:192.168.201.0  Mask:255.255.255.255
            UP RUNNING NOARP  MTU:1440  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1
            RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
  
  
  root@iZ2ze8ctk2q17u029a8wcoZ:~# route -n
  Kernel IP routing table
  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  0.0.0.0         59.110.35.247   0.0.0.0         UG    0      0        0 eth1
  10.27.216.0     0.0.0.0         255.255.252.0   U     0      0        0 eth0
  10.30.0.0       10.27.219.247   255.255.0.0     UG    0      0        0 eth0
  10.32.0.0       0.0.0.0         255.240.0.0     U     0      0        0 weave
  59.110.32.0     0.0.0.0         255.255.252.0   U     0      0        0 eth1
  100.64.0.0      10.27.219.247   255.192.0.0     UG    0      0        0 eth0
  172.16.0.0      10.27.219.247   255.240.0.0     UG    0      0        0 eth0
  172.17.0.0      0.0.0.0         255.255.255.0   U     0      0        0 docker0
  192.168.201.0   0.0.0.0         255.255.255.192 U     0      0        0 *
  
  root@iZ2ze8ctk2q17u029a8wcoZ:~# ip route list
  default via 59.110.35.247 dev eth1
  10.27.216.0/22 dev eth0  proto kernel  scope link  src 10.27.219.50
  10.30.0.0/16 via 10.27.219.247 dev eth0
  10.32.0.0/12 dev weave  proto kernel  scope link  src 10.32.0.1
  59.110.32.0/22 dev eth1  proto kernel  scope link  src 59.110.32.199
  100.64.0.0/10 via 10.27.219.247 dev eth0
  172.16.0.0/12 via 10.27.219.247 dev eth0
  172.17.0.0/24 dev docker0  proto kernel  scope link  src 172.17.0.1 linkdown
  blackhole 192.168.201.0/26  proto bird
  
  // NOTE: 10.30.0.0/16 via 10.27.219.247 dev eth0
  // this rule is important, the worker node's ip is 10.30.xx.xx. If I delete this rule, I cannot ping worker node.
  // this rule is 10.0.0.0/8 via 10.27.219.247 dev eth0 by default, I changed it to the above.
  
  
  root@iZ2ze8ctk2q17u029a8wcoZ:~# iptables -t nat -nvL
  Chain PREROUTING (policy ACCEPT 3 packets, 180 bytes)
   pkts bytes target     prot opt in     out     source               destination
  20976 1250K cali-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:6gwbT8clXdHdC1b1 */
  21016 1252K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
  20034 1193K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
  
  Chain INPUT (policy ACCEPT 3 packets, 180 bytes)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain OUTPUT (policy ACCEPT 4 packets, 240 bytes)
   pkts bytes target     prot opt in     out     source               destination
   109K 6580K cali-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:tVnHkvAo15HuiPy0 */
   111K 6738K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
   1263 75780 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL
  
  Chain POSTROUTING (policy ACCEPT 4 packets, 240 bytes)
   pkts bytes target     prot opt in     out     source               destination
  86584 5235K cali-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:O3lYWMrLQYEMJtB5 */
      0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/24        0.0.0.0/0
  3982K  239M KUBE-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes postrouting rules */
  28130 1704K WEAVE      all  --  *      *       0.0.0.0/0            0.0.0.0/0
  
  Chain DOCKER (2 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
  
  Chain KUBE-MARK-DROP (0 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x8000
  
  Chain KUBE-MARK-MASQ (5 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000
  
  Chain KUBE-NODEPORTS (1 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain KUBE-POSTROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
  
  Chain KUBE-SEP-2VS52M6CEWASZVOP (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.31.149       0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:192.168.31.149:53
  
  Chain KUBE-SEP-3XQHSFTDAPNNNDX3 (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.31.150       0.0.0.0/0            /* kube-system/heapster: */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/heapster: */ tcp to:192.168.31.150:8082
  
  Chain KUBE-SEP-CH7KJM5XKO5WGA6D (2 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       10.27.219.50         0.0.0.0/0            /* default/kubernetes:https */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ recent: SET name: KUBE-SEP-CH7KJM5XKO5WGA6D side: source mask: 255.255.255.255 tcp to:10.27.219.50:6443
  
  Chain KUBE-SEP-X3WTOMIYJNS7APAN (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.31.149       0.0.0.0/0            /* kube-system/kube-dns:dns */
      0     0 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:192.168.31.149:53
  
  Chain KUBE-SEP-YDCHDMTZNPMRRKCX (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       10.27.219.50         0.0.0.0/0            /* kube-system/calico-etcd: */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/calico-etcd: */ tcp to:10.27.219.50:6666
  
  Chain KUBE-SERVICES (2 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SVC-NPX46M4PTMTKRN6Y  tcp  --  *      *       0.0.0.0/0            10.96.0.1            /* default/kubernetes:https cluster IP */ tcp dpt:443
      0     0 KUBE-SVC-TCOU7JCQXEZGVUNU  udp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
      0     0 KUBE-SVC-ERIFXISQEP7F7OF4  tcp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
      0     0 KUBE-SVC-NTYB37XIWATNM25Y  tcp  --  *      *       0.0.0.0/0            10.96.232.136        /* kube-system/calico-etcd: cluster IP */ tcp dpt:6666
      0     0 KUBE-SVC-BJM46V3U5RZHCFRZ  tcp  --  *      *       0.0.0.0/0            10.96.181.180        /* kube-system/heapster: cluster IP */ tcp dpt:80
      7   420 KUBE-NODEPORTS  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL
  
  Chain KUBE-SVC-BJM46V3U5RZHCFRZ (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-3XQHSFTDAPNNNDX3  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/heapster: */
  
  Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-2VS52M6CEWASZVOP  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
  
  Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-CH7KJM5XKO5WGA6D  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ recent: CHECK seconds: 10800 reap name: KUBE-SEP-CH7KJM5XKO5WGA6D side: source mask: 255.255.255.255
      0     0 KUBE-SEP-CH7KJM5XKO5WGA6D  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */
  
  Chain KUBE-SVC-NTYB37XIWATNM25Y (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-YDCHDMTZNPMRRKCX  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/calico-etcd: */
  
  Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-X3WTOMIYJNS7APAN  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */
  
  Chain WEAVE (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 RETURN     all  --  *      *       10.32.0.0/12         224.0.0.0/4
      1    93 MASQUERADE  all  --  *      *      !10.32.0.0/12         10.32.0.0/12
      0     0 MASQUERADE  all  --  *      *       10.32.0.0/12        !10.32.0.0/12
  
  Chain cali-OUTPUT (1 references)
   pkts bytes target     prot opt in     out     source               destination
   109K 6580K cali-fip-dnat  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:GBTAv2p5CwevEyJm */
  
  Chain cali-POSTROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination
   109K 6571K cali-fip-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:Z-c7XtVd2Bq7s_hA */
   109K 6571K cali-nat-outgoing  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:nYKhEzDlr11Jccal */
      0     0 MASQUERADE  all  --  *      tunl0   0.0.0.0/0            0.0.0.0/0            /* cali:JHlpT-eSqR1TvyYm */ ADDRTYPE match src-type !LOCAL limit-out ADDRTYPE match src-type LOCAL
  
  Chain cali-PREROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination
  20976 1250K cali-fip-dnat  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:r6XmIziWUJsdOK6Z */
  
  Chain cali-fip-dnat (2 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain cali-fip-snat (1 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain cali-nat-outgoing (1 references)
   pkts bytes target     prot opt in     out     source               destination
      4   376 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:Wd76s91357Uv7N3v */ match-set cali4-masq-ipam-pools src ! match-set cali4-all-ipam-pools dst
  </code></pre>
  
  <p>Worker Node Info:
  internal ip: 10.30.248.80</p>
  
  <pre><code>ifconfig
  
  docker0   Link encap:Ethernet  HWaddr 02:42:58:2b:b5:39
            inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.255.0
            UP BROADCAST MULTICAST  MTU:1500  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:0
            RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
  
  eth0      Link encap:Ethernet  HWaddr 00:16:3e:2e:3d:fd
            inet addr:10.30.248.80  Bcast:10.30.251.255  Mask:255.255.252.0
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:3856596 errors:0 dropped:0 overruns:0 frame:0
            TX packets:4253613 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:827402268 (827.4 MB)  TX bytes:510838231 (510.8 MB)
  
  eth1      Link encap:Ethernet  HWaddr 00:16:3e:2c:db:d1
            inet addr:47.93.161.177  Bcast:47.93.163.255  Mask:255.255.252.0
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:890451 errors:0 dropped:0 overruns:0 frame:0
            TX packets:825607 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:1695352720 (1.6 GB)  TX bytes:62341312 (62.3 MB)
  
  tunl0     Link encap:IPIP Tunnel  HWaddr
            inet addr:192.168.31.128  Mask:255.255.255.255
            UP RUNNING NOARP  MTU:1440  Metric:1
            RX packets:0 errors:0 dropped:0 overruns:0 frame:0
            TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1
            RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
  
  
  root@iZ2zegw6nmd5t5qxy35lh0Z:~# route -n
  Kernel IP routing table
  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  0.0.0.0         47.93.163.247   0.0.0.0         UG    0      0        0 eth1
  10.0.0.0        10.30.251.247   255.0.0.0       UG    0      0        0 eth0
  10.30.248.0     0.0.0.0         255.255.252.0   U     0      0        0 eth0
  47.93.160.0     0.0.0.0         255.255.252.0   U     0      0        0 eth1
  100.64.0.0      10.30.251.247   255.192.0.0     UG    0      0        0 eth0
  172.16.0.0      10.30.251.247   255.240.0.0     UG    0      0        0 eth0
  172.17.0.0      0.0.0.0         255.255.255.0   U     0      0        0 docker0
  192.168.31.128  0.0.0.0         255.255.255.192 U     0      0        0 *
  192.168.31.149  0.0.0.0         255.255.255.255 UH    0      0        0 cali3567b3362cc
  192.168.31.150  0.0.0.0         255.255.255.255 UH    0      0        0 cali9d04015b0e7
  
  root@iZ2zegw6nmd5t5qxy35lh0Z:~# ip route list
  default via 47.93.163.247 dev eth1
  10.0.0.0/8 via 10.30.251.247 dev eth0
  10.30.248.0/22 dev eth0  proto kernel  scope link  src 10.30.248.80
  47.93.160.0/22 dev eth1  proto kernel  scope link  src 47.93.161.177
  100.64.0.0/10 via 10.30.251.247 dev eth0
  172.16.0.0/12 via 10.30.251.247 dev eth0
  172.17.0.0/24 dev docker0  proto kernel  scope link  src 172.17.0.1 linkdown
  blackhole 192.168.31.128/26  proto bird
  192.168.31.149 dev cali3567b3362cc  scope link
  192.168.31.150 dev cali9d04015b0e7  scope link
  
  // NOTE: 10.0.0.0/8 via 10.30.251.247 dev eth0
  // I didn't change this one. So it is default now.
  
  
  root@iZ2zegw6nmd5t5qxy35lh0Z:~# iptables -t nat -nvL
  Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
   pkts bytes target     prot opt in     out     source               destination
   3524  263K cali-PREROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:6gwbT8clXdHdC1b1 */
   3527  263K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
   1031 53882 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
  
  Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain OUTPUT (policy ACCEPT 4 packets, 240 bytes)
   pkts bytes target     prot opt in     out     source               destination
  84174 5099K cali-OUTPUT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:tVnHkvAo15HuiPy0 */
  85201 5163K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
      0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL
  
  Chain POSTROUTING (policy ACCEPT 7 packets, 420 bytes)
   pkts bytes target     prot opt in     out     source               destination
  76279 4644K cali-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:O3lYWMrLQYEMJtB5 */
      0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/24        0.0.0.0/0
  87179 5342K KUBE-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes postrouting rules */
  43815 2646K WEAVE      all  --  *      *       0.0.0.0/0            0.0.0.0/0
  
  Chain DOCKER (2 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
  
  Chain KUBE-MARK-DROP (0 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x8000
  
  Chain KUBE-MARK-MASQ (5 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000
  
  Chain KUBE-NODEPORTS (1 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain KUBE-POSTROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
  
  Chain KUBE-SEP-2VS52M6CEWASZVOP (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.31.149       0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:192.168.31.149:53
  
  Chain KUBE-SEP-3XQHSFTDAPNNNDX3 (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.31.150       0.0.0.0/0            /* kube-system/heapster: */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/heapster: */ tcp to:192.168.31.150:8082
  
  Chain KUBE-SEP-CH7KJM5XKO5WGA6D (2 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       10.27.219.50         0.0.0.0/0            /* default/kubernetes:https */
      3   180 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ recent: SET name: KUBE-SEP-CH7KJM5XKO5WGA6D side: source mask: 255.255.255.255 tcp to:10.27.219.50:6443
  
  Chain KUBE-SEP-X3WTOMIYJNS7APAN (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       192.168.31.149       0.0.0.0/0            /* kube-system/kube-dns:dns */
      0     0 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:192.168.31.149:53
  
  Chain KUBE-SEP-YDCHDMTZNPMRRKCX (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-MARK-MASQ  all  --  *      *       10.27.219.50         0.0.0.0/0            /* kube-system/calico-etcd: */
      0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/calico-etcd: */ tcp to:10.27.219.50:6666
  
  Chain KUBE-SERVICES (2 references)
   pkts bytes target     prot opt in     out     source               destination
      3   180 KUBE-SVC-NPX46M4PTMTKRN6Y  tcp  --  *      *       0.0.0.0/0            10.96.0.1            /* default/kubernetes:https cluster IP */ tcp dpt:443
      0     0 KUBE-SVC-TCOU7JCQXEZGVUNU  udp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
      0     0 KUBE-SVC-ERIFXISQEP7F7OF4  tcp  --  *      *       0.0.0.0/0            10.96.0.10           /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
      0     0 KUBE-SVC-NTYB37XIWATNM25Y  tcp  --  *      *       0.0.0.0/0            10.96.232.136        /* kube-system/calico-etcd: cluster IP */ tcp dpt:6666
      0     0 KUBE-SVC-BJM46V3U5RZHCFRZ  tcp  --  *      *       0.0.0.0/0            10.96.181.180        /* kube-system/heapster: cluster IP */ tcp dpt:80
      0     0 KUBE-NODEPORTS  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL
  
  Chain KUBE-SVC-BJM46V3U5RZHCFRZ (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-3XQHSFTDAPNNNDX3  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/heapster: */
  
  Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-2VS52M6CEWASZVOP  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
  
  Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
   pkts bytes target     prot opt in     out     source               destination
      3   180 KUBE-SEP-CH7KJM5XKO5WGA6D  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ recent: CHECK seconds: 10800 reap name: KUBE-SEP-CH7KJM5XKO5WGA6D side: source mask: 255.255.255.255
      0     0 KUBE-SEP-CH7KJM5XKO5WGA6D  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */
  
  Chain KUBE-SVC-NTYB37XIWATNM25Y (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-YDCHDMTZNPMRRKCX  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/calico-etcd: */
  
  Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
   pkts bytes target     prot opt in     out     source               destination
      0     0 KUBE-SEP-X3WTOMIYJNS7APAN  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */
  
  Chain WEAVE (1 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain cali-OUTPUT (1 references)
   pkts bytes target     prot opt in     out     source               destination
  84174 5099K cali-fip-dnat  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:GBTAv2p5CwevEyJm */
  
  Chain cali-POSTROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination
  86501 5298K cali-fip-snat  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:Z-c7XtVd2Bq7s_hA */
  86501 5298K cali-nat-outgoing  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:nYKhEzDlr11Jccal */
      0     0 MASQUERADE  all  --  *      tunl0   0.0.0.0/0            0.0.0.0/0            /* cali:JHlpT-eSqR1TvyYm */ ADDRTYPE match src-type !LOCAL limit-out ADDRTYPE match src-type LOCAL
  
  Chain cali-PREROUTING (1 references)
   pkts bytes target     prot opt in     out     source               destination
   3524  263K cali-fip-dnat  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:r6XmIziWUJsdOK6Z */
  
  Chain cali-fip-dnat (2 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain cali-fip-snat (1 references)
   pkts bytes target     prot opt in     out     source               destination
  
  Chain cali-nat-outgoing (1 references)
   pkts bytes target     prot opt in     out     source               destination
     29  1726 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:Wd76s91357Uv7N3v */ match-set cali4-masq-ipam-pools src ! match-set cali4-all-ipam-pools dst
  </code></pre>