<p>I'm getting a strange <code>400</code> error when I try to connect to an <code>Istio Gateway</code> when behind an <code>AWS</code> load balancer. </p>

<p>I don't see any activity in the <code>istio-ingresgateway</code> logs (even with debug settings on), but when I run <code>sysdig</code> on the <code>ingressgateway</code> pod, I see weird semi-random text with often with <code>"QUIT !T"</code> in it. </p>

<p>I get this when I try to make an <code>http</code> request via a browser or <code>curl</code> from outside the cluster. </p>

<p>The same <code>Istio</code> configuration works when I try to make the request in <code>minikube</code> or in <code>Azure</code>. </p>

<p>I'm also able to use the same <code>AWS</code> lb to point to a <code>Nginx</code> ingress controller and it works just fine.</p>

<pre><code>sudo sysdig -s2000 -A -c echo_fds fd.ip=10.1.2.3
</code></pre>

<p>Sometimes there is no GET request in the output</p>

<pre><code>------ Read 100B from   10.1.1.3:44404-&gt;10.1.2.3:80 (envoy)




QUIT
!T
C
ct&gt;
------ Write 66B to   10.1.1.3:44404-&gt;10.1.2.3:80 (envoy)

HTTP/1.1 400 Bad Request
content-length: 0
connection: close
</code></pre>

<p>And, sometimes this happens right before the <code>GET</code> request</p>

<pre><code>------ Read 3.39KB from   10.1.1.3:35430-&gt;10.1.2.3:80 (envoy)




QUIT
!T
C
atfI&gt;GET /myapp/ HTTP/1.1
</code></pre>

<p>I'm wondering if the weird characters are causing the <code>envoy</code> routes not to match, but I have no idea where this could be coming from. </p>

<p>Any advice as to what this might be? </p>

<p>Any general strategies for debugging <code>Istio</code> ingress? </p>

<p>Any help is much appreciated. </p>

<p>So I found the answer to this question. The garbage in the request was a read herring. </p>

<p>A little more info about the setup:</p>

<p>The AWS load balancer was terminating TLS, so all the traffic behind it was going over the http port 31380. I was already passing the <code>X-Forwarded-Proto: https</code> header via the istio VirtualService setting, so the backend applications could assume that the traffic was already-terminated TLS traffic.</p>

<p>The solution:</p>

<p>The issue in my case was that the AWS Target Group had the <code>Proxy protocol v2: Enabled</code> set on it.  Turning this off solved the issue.</p>
