<p>Istio on Kubernetes injects an Envoy sidecar to run alongside Pods and implement a service mesh, however Istio itself <a href="https://istio.io/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations" rel="nofollow noreferrer">cannot ensure traffic does not bypass this proxy</a>; if that happens Istio security policy is no longer applied.</p>

<p>Therefore, I am trying to understand all ways in which this bypass could happen (assuming Envoy itself hasn't been compromised) and find ways to prevent them so that TCP traffic originating from a Pod's network namespace is guaranteed to have gone through Envoy (or at least is much more likely to have done):</p>

<ol>
<li>Since (at the time of writing) Envoy does not support UDP (it's <a href="https://github.com/envoyproxy/envoy/pull/9046" rel="nofollow noreferrer">nearly there</a>), UDP traffic won't be proxied, so use <a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/" rel="nofollow noreferrer">NetworkPolicy</a> to ensure only TCP traffic is allowed to/from the Pod (e.g. to avoid TCP traffic being tunnelled out via a VPN over UDP)</li>
<li>Drop NET_ADMIN capability to prevent the Pod from reconfiguring the IPTables rules in its network namespace that capture traffic</li>
<li>Drop NET_RAW capability to prevent the Pod from opening raw sockets and bypassing the netfilter hook points that IPTables makes use of</li>
</ol>

<p>The only other attack vector I know of would be a kernel vulnerability - are there any others? Maybe there are other L3/4 protocols that IPTables doesn't recognise or ignores?</p>

<p>I understand that <a href="https://www.youtube.com/watch?v=ER9eIXL2_14&amp;t=17m22s" rel="nofollow noreferrer">eBPF and Cilium</a> could be used to enforce this interception at the socket level, but I am interested in the case of using vanilla Istio on Kubernetes.</p>

<p>EDIT: I am also assuming the workload does not have Kubernetes API server access</p>

dippynark
<p>Envoy is not designed to be used as a firewall. Service meshes that rely on it such as Istio or Cilium only consider it a bug if you can bypass the policies on the receiving end.</p>

<p>For example, any pod can trivially bypass any Istio or Cilium policies by terminating its own Envoy with <code>curl localhost:15000/quitquitquit</code> and starting a custom proxy on port 15001 that allows everything before Envoy is restarted.</p>

<p>You can patch up that particular hole, but since resisting such attacks <em>is not a design goal</em> for the service meshes, there are probably dozens of other ways to accomplish the same thing. New ways bypass these policies may also be added in subsequent releases.</p>

<p>If you want your security policies to be actually enforced on the end that initiates the connection and not only on the receiving end, consider using a network policy implementation for which it <em>is</em> a design goal, such as Calico.</p>
