<p>We have set up Istio, and we are using ISTIO ingress gateway for inbound traffic. We have set up TLS for TCP port. Sample code can be found <a href="https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/" rel="nofollow noreferrer">here</a>.</p>
<p>We also enabled logs by following <a href="https://istio.io/latest/docs/tasks/observability/logs/access-log/" rel="nofollow noreferrer">this ISTIO guide</a></p>
<p>We tested the TLS connection using openssl and it works fine.</p>
<p>However, when we try to connect from an application, the TLS negotiation fails. I have provided more details with wireshark <a href="https://devops.stackexchange.com/questions/12019/tls-handshake-failure">here</a></p>
<p>We would like to get logs from ISTIO on the TLS negotiation ... and find why it fails.</p>
<p>Istio Gateway YAML</p>
<pre><code>apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: dremio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    tls:
      httpsRedirect: true
    hosts:
    - testdomain.net
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: testdomain-credentials
    hosts:
    - testdomain.net
  - port:
      number: 31020
      name: odbc-dremio-tls
      protocol: tls
    tls:
      mode: SIMPLE
      minProtocolVersion: TLSV1_0
      maxProtocolVersion: TLSV1_3
      credentialName: testdomain-credentials
    hosts:
    - testdomain.net
</code></pre>
<p>Virtual Service</p>
<pre><code>apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: dremio
spec:
  hosts:
  - testdomain.net
  gateways:
  - dremio-gateway
  http:
  - match:
    - port: 443
    - port: 80
    route:
    - destination:
        host: dremio-client
        port:
          number: 9047
  tcp:
  - match:
    - port: 31020
    route:
    - destination:
        host: dremio-client
        port:
          number: 31010
</code></pre>
<p>Partial Config Dump</p>
<pre><code>{
     &quot;name&quot;: &quot;0.0.0.0_31020&quot;,
     &quot;active_state&quot;: {
      &quot;version_info&quot;: &quot;2020-07-21T12:11:49Z/9&quot;,
      &quot;listener&quot;: {
       &quot;@type&quot;: &quot;type.googleapis.com/envoy.api.v2.Listener&quot;,
       &quot;name&quot;: &quot;0.0.0.0_31020&quot;,
       &quot;address&quot;: {
        &quot;socket_address&quot;: {
         &quot;address&quot;: &quot;0.0.0.0&quot;,
         &quot;port_value&quot;: 31020
        }
       },
       &quot;filter_chains&quot;: [
        {
         &quot;filter_chain_match&quot;: {
          &quot;server_names&quot;: [
           &quot;testdomain.net&quot;
          ]
         },
         &quot;filters&quot;: [
          {
           &quot;name&quot;: &quot;istio.stats&quot;,
           &quot;typed_config&quot;: {
            &quot;@type&quot;: &quot;type.googleapis.com/udpa.type.v1.TypedStruct&quot;,
            &quot;type_url&quot;: &quot;type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm&quot;,
            &quot;value&quot;: {
             &quot;config&quot;: {
              &quot;root_id&quot;: &quot;stats_outbound&quot;,
              &quot;vm_config&quot;: {
               &quot;vm_id&quot;: &quot;tcp_stats_outbound&quot;,
               &quot;runtime&quot;: &quot;envoy.wasm.runtime.null&quot;,
               &quot;code&quot;: {
                &quot;local&quot;: {
                 &quot;inline_string&quot;: &quot;envoy.wasm.stats&quot;
                }
               }
              },
              &quot;configuration&quot;: &quot;{\n  \&quot;debug\&quot;: \&quot;false\&quot;,\n  \&quot;stat_prefix\&quot;: \&quot;istio\&quot;\n}\n&quot;
             }
            }
           }
          },
          {
           &quot;name&quot;: &quot;envoy.tcp_proxy&quot;,
           &quot;typed_config&quot;: {
            &quot;@type&quot;: &quot;type.googleapis.com/envoy.config.filter.network.tcp_proxy.v2.TcpProxy&quot;,
            &quot;stat_prefix&quot;: &quot;outbound|31010||dremio-client.dremio.svc.cluster.local&quot;,
            &quot;cluster&quot;: &quot;outbound|31010||dremio-client.dremio.svc.cluster.local&quot;,
            &quot;access_log&quot;: [
             {
              &quot;name&quot;: &quot;envoy.file_access_log&quot;,
              &quot;typed_config&quot;: {
               &quot;@type&quot;: &quot;type.googleapis.com/envoy.config.accesslog.v2.FileAccessLog&quot;,
               &quot;path&quot;: &quot;/dev/stdout&quot;,
               &quot;format&quot;: &quot;[%START_TIME%] \&quot;%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\&quot; %RESPONSE_CODE% %RESPONSE_FLAGS% \&quot;%DYNAMIC_METADATA(istio.mixer:status)%\&quot; \&quot;%UPSTREAM_TRANSPORT_FAILURE_REASON%\&quot; %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \&quot;%REQ(X-FORWARDED-FOR)%\&quot; \&quot;%REQ(USER-AGENT)%\&quot; \&quot;%REQ(X-REQUEST-ID)%\&quot; \&quot;%REQ(:AUTHORITY)%\&quot; \&quot;%UPSTREAM_HOST%\&quot; %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%\n&quot;
              }
             }
            ]
           }
          }
         ],
         &quot;transport_socket&quot;: {
          &quot;name&quot;: &quot;envoy.transport_sockets.tls&quot;,
          &quot;typed_config&quot;: {
           &quot;@type&quot;: &quot;type.googleapis.com/envoy.api.v2.auth.DownstreamTlsContext&quot;,
           &quot;common_tls_context&quot;: {
            &quot;tls_params&quot;: {
             &quot;tls_minimum_protocol_version&quot;: &quot;TLSv1_0&quot;,
             &quot;tls_maximum_protocol_version&quot;: &quot;TLSv1_3&quot;
            },
            &quot;alpn_protocols&quot;: [
             &quot;h2&quot;,
             &quot;http/1.1&quot;
            ],
            &quot;tls_certificate_sds_secret_configs&quot;: [
             {
              &quot;name&quot;: &quot;testdomain-credentials&quot;,
              &quot;sds_config&quot;: {
               &quot;api_config_source&quot;: {
                &quot;api_type&quot;: &quot;GRPC&quot;,
                &quot;grpc_services&quot;: [
                 {
                  &quot;google_grpc&quot;: {
                   &quot;target_uri&quot;: &quot;unix:/var/run/ingress_gateway/sds&quot;,
                   &quot;stat_prefix&quot;: &quot;sdsstat&quot;
                  }
                 }
                ]
               }
              }
             }
            ]
           },
           &quot;require_client_certificate&quot;: false
          }
         }
        }
       ],
       &quot;listener_filters&quot;: [
        {
         &quot;name&quot;: &quot;envoy.listener.tls_inspector&quot;,
         &quot;typed_config&quot;: {
          &quot;@type&quot;: &quot;type.googleapis.com/envoy.config.filter.listener.tls_inspector.v2.TlsInspector&quot;
         }
        }
       ],
       &quot;traffic_direction&quot;: &quot;OUTBOUND&quot;
      },
      &quot;last_updated&quot;: &quot;2020-07-21T12:11:50.303Z&quot;
     }
    }
</code></pre>
<p>By enabling tracing on Envoy conn_handler, We can see the following message:</p>
<blockquote>
<p>closing connection: no matching filter chain found</p>
</blockquote>

Abba
<p>After getting the message of no matching filter chain, I found the filter chain for the port 31020 with the domain that I have provided in my Gateway config. It looks like while connecting my application(ODBC), the host was not being provided.</p>
<p>The solution is simply to replace the host domain by '*'</p>
<pre><code>apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: dremio-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    tls:
      httpsRedirect: true
    hosts:
    - testdomain.net
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: testdomain-credentials
    hosts:
    - testdomain.net
  - port:
      number: 31020
      name: odbc-dremio-tls
      protocol: tls
    tls:
      mode: SIMPLE
      minProtocolVersion: TLSV1_0
      maxProtocolVersion: TLSV1_3
      credentialName: testdomain-credentials
    hosts:
    - '*'
</code></pre>
