<p>I'm testing <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/deploying-gateways" rel="nofollow noreferrer">Gateway API</a> with GKE (version 1.21.11-gke.1100). I'm using <code>gatewayClassName: gke-l7-rilb</code> for a  Gateway with TLS between the client and the gateway. HTTPS is working perfectly between the client and the loadbalancer using a managed regional SSL certificate .</p>
<p>I have 2 Httproutes referencing 2 kube services (<code>backendRefs</code>). One service is accessible through HTTP and the other through HTTPS (argo-server service from Argo Workflows project if it may helps).</p>
<p>When I create the httproute referencing the service using HTTP, the GCP load balancer backend service is created and working without any problem (Healthy).</p>
<p>But when I create the httproute referencing argo-service, a GCP load balancer backend service is created but not working (not healthy) with an endpoint protocol set to HTTP rather than HTTPS. You should know that I made sure to add to argo-server service the annotation <code>cloud.google.com/app-protocols: '{&quot;web&quot;:&quot;HTTPS&quot;}'</code> to enable HTTPS between the load balancer and argo-server application.</p>
<p>If I create the same geatway api configuration using ingress resource and the same argo service definition, endpoint protocol ( of the GCP load balancer backend service )is set correctly to HTTPS and is perfectly healthy and working.</p>
<p>As if httproute of the gateway API the GKE gateway controller are not taking into consideration the  <code>cloud.google.com/app-protocols</code> service annotation although it is mentioned that it's relevant for the gateway API <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/gatewayclass-capabilities#service" rel="nofollow noreferrer">here</a>.</p>
<p>Edit 1: added yaml files</p>
<ul>
<li>Gateway:</li>
</ul>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: gateway.networking.k8s.io/v1alpha2
kind: Gateway
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {&quot;apiVersion&quot;:&quot;gateway.networking.k8s.io/v1alpha2&quot;,&quot;kind&quot;:&quot;Gateway&quot;,&quot;metadata&quot;:{&quot;annotations&quot;:{},&quot;labels&quot;:{&quot;app.kubernetes.io/managed-by&quot;:&quot;gcp-cloud-build-deploy&quot;},&quot;name&quot;:&quot;regional-internal-https&quot;,&quot;namespace&quot;:&quot;exposition&quot;},&quot;spec&quot;:{&quot;addresses&quot;:[{&quot;type&quot;:&quot;NamedAddress&quot;,&quot;value&quot;:&quot;dev-gateway-internal-lb-static-ip&quot;}],&quot;gatewayClassName&quot;:&quot;gke-l7-rilb&quot;,&quot;listeners&quot;:[{&quot;allowedRoutes&quot;:{&quot;kinds&quot;:[{&quot;kind&quot;:&quot;HTTPRoute&quot;}],&quot;namespaces&quot;:{&quot;from&quot;:&quot;Selector&quot;,&quot;selector&quot;:{&quot;matchLabels&quot;:{&quot;exposed&quot;:&quot;true&quot;}}}},&quot;name&quot;:&quot;https&quot;,&quot;port&quot;:443,&quot;protocol&quot;:&quot;HTTPS&quot;,&quot;tls&quot;:{&quot;mode&quot;:&quot;Terminate&quot;,&quot;options&quot;:{&quot;networking.gke.io/pre-shared-certs&quot;:&quot;plat-dev-europe-west1&quot;}}}]}}
    networking.gke.io/addresses: &quot;&quot;
    networking.gke.io/backend-services: gkegw1-bkib-argo-argo-server-2746-8ktcvo8d0ktp,
      gkegw1-bkib-demo-application-demo-service-80-y5bgcnm71kjv, gkegw1-bkib-exposition-gw-serve404-80-pciznuyt569p
    networking.gke.io/firewalls: &quot;&quot;
    networking.gke.io/forwarding-rules: gkegw1-bkib-exposition-regional-internal-https-tqsh4njw7io8
    networking.gke.io/health-checks: gkegw1-bkib-argo-argo-server-2746-8ktcvo8d0ktp,
      gkegw1-bkib-demo-application-demo-service-80-y5bgcnm71kjv, gkegw1-bkib-exposition-gw-serve404-80-pciznuyt569p
    networking.gke.io/last-reconcile-time: &quot;2022-06-16T15:57:45Z&quot;
    networking.gke.io/ssl-certificates: &quot;&quot;
    networking.gke.io/target-proxies: gkegw1-bkib-exposition-regional-internal-https-tqsh4njw7io8
    networking.gke.io/url-maps: gkegw1-bkib-exposition-regional-internal-https-tqsh4njw7io8
  creationTimestamp: &quot;2022-06-15T08:28:20Z&quot;
  finalizers:
  - gateway.finalizer.networking.gke.io
  generation: 1
  labels:
    app.kubernetes.io/managed-by: gcp-cloud-build-deploy
  managedFields:
  - apiVersion: gateway.networking.k8s.io/v1alpha2
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/managed-by: {}
      f:spec:
        .: {}
        f:addresses: {}
        f:gatewayClassName: {}
        f:listeners:
          .: {}
          k:{&quot;name&quot;:&quot;https&quot;}:
            .: {}
            f:allowedRoutes:
              .: {}
              f:kinds: {}
              f:namespaces:
                .: {}
                f:from: {}
                f:selector:
                  .: {}
                  f:matchLabels:
                    .: {}
                    f:exposed: {}
            f:name: {}
            f:port: {}
            f:protocol: {}
            f:tls:
              .: {}
              f:mode: {}
              f:options:
                .: {}
                f:networking.gke.io/pre-shared-certs: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: &quot;2022-06-15T08:28:20Z&quot;
  - apiVersion: gateway.networking.k8s.io/v1alpha2
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:networking.gke.io/addresses: {}
          f:networking.gke.io/backend-services: {}
          f:networking.gke.io/firewalls: {}
          f:networking.gke.io/forwarding-rules: {}
          f:networking.gke.io/health-checks: {}
          f:networking.gke.io/last-reconcile-time: {}
          f:networking.gke.io/ssl-certificates: {}
          f:networking.gke.io/target-proxies: {}
          f:networking.gke.io/url-maps: {}
        f:finalizers:
          .: {}
          v:&quot;gateway.finalizer.networking.gke.io&quot;: {}
      f:status:
        f:addresses: {}
    manager: GoogleGKEGatewayController
    operation: Update
    time: &quot;2022-06-15T08:30:16Z&quot;
  name: regional-internal-https
  namespace: exposition
  resourceVersion: &quot;42337844&quot;
  uid: 59333aea-1a79-4e9b-afbc-595ae9ccdfd7
spec:
  addresses:
  - type: NamedAddress
    value: dev-gateway-internal-lb-static-ip
  gatewayClassName: gke-l7-rilb
  listeners:
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: HTTPRoute
      namespaces:
        from: Selector
        selector:
          matchLabels:
            exposed: &quot;true&quot;
    name: https
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      options:
        networking.gke.io/pre-shared-certs: plat-dev-europe-west1
status:
  addresses:
  - type: IPAddress
    value: 10.163.112.28
  conditions:
  - lastTransitionTime: &quot;1970-01-01T00:00:00Z&quot;
    message: Waiting for controller
    reason: NotReconciled
    status: Unknown
    type: Scheduled

</code></pre>
<ul>
<li>Httproute:</li>
</ul>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: gateway.networking.k8s.io/v1alpha2
kind: HTTPRoute
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {&quot;apiVersion&quot;:&quot;gateway.networking.k8s.io/v1alpha2&quot;,&quot;kind&quot;:&quot;HTTPRoute&quot;,&quot;metadata&quot;:{&quot;annotations&quot;:{},&quot;labels&quot;:{&quot;app.kubernetes.io/managed-by&quot;:&quot;gcp-cloud-build-deploy&quot;},&quot;name&quot;:&quot;argo-server&quot;,&quot;namespace&quot;:&quot;argo&quot;},&quot;spec&quot;:{&quot;hostnames&quot;:[&quot;argo-server.plat.dev.df.gcp.corp.modified.com&quot;],&quot;parentRefs&quot;:[{&quot;kind&quot;:&quot;Gateway&quot;,&quot;name&quot;:&quot;regional-internal-https&quot;,&quot;namespace&quot;:&quot;exposition&quot;}],&quot;rules&quot;:[{&quot;backendRefs&quot;:[{&quot;name&quot;:&quot;argo-server&quot;,&quot;port&quot;:2746}]}]}}
  creationTimestamp: &quot;2022-06-15T12:27:04Z&quot;
  generation: 1
  labels:
    app.kubernetes.io/managed-by: gcp-cloud-build-deploy
  managedFields:
    - apiVersion: gateway.networking.k8s.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
          f:labels:
            .: {}
            f:app.kubernetes.io/managed-by: {}
        f:spec:
          .: {}
          f:hostnames: {}
          f:parentRefs: {}
          f:rules: {}
      manager: kubectl-client-side-apply
      operation: Update
      time: &quot;2022-06-15T12:27:04Z&quot;
    - apiVersion: gateway.networking.k8s.io/v1alpha2
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          .: {}
          f:parents: {}
      manager: GoogleGKEGatewayController
      operation: Update
      time: &quot;2022-06-15T12:29:02Z&quot;
  name: argo-server
  namespace: argo
  resourceVersion: &quot;42362026&quot;
  uid: 981ce997-c574-4878-bec1-b03c7707838c
spec:
  hostnames:
    - argo-server.plat.dev.df.gcp.corp.modified.com
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: regional-internal-https
      namespace: exposition
  rules:
    - backendRefs:
        - group: &quot;&quot;
          kind: Service
          name: argo-server
          port: 2746
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /
status:
  parents:
    - conditions:
        - lastTransitionTime: &quot;2022-06-16T17:00:11Z&quot;
          message: &quot;&quot;
          reason: RouteAccepted
          status: &quot;True&quot;
          type: Accepted
        - lastTransitionTime: &quot;2022-06-16T17:00:11Z&quot;
          message: &quot;&quot;
          reason: ReconciliationSucceeded
          status: &quot;True&quot;
          type: Reconciled
      controllerName: networking.gke.io/gateway
      parentRef:
        group: gateway.networking.k8s.io
        kind: Gateway
        name: regional-internal-https
        namespace: exposition

</code></pre>
<ul>
<li>Service:</li>
</ul>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/app-protocols: '{&quot;web&quot;:&quot;HTTPS&quot;}'
    cloud.google.com/backend-config: '{&quot;default&quot;: &quot;argo-server-backendconfig&quot;}'
    cloud.google.com/neg: '{&quot;exposed_ports&quot;:{&quot;2746&quot;:{}}}'
    cloud.google.com/neg-status: '{&quot;network_endpoint_groups&quot;:{&quot;2746&quot;:&quot;k8s1-f83345f9-argo-argo-server-2746-4d39c835&quot;},&quot;zones&quot;:[&quot;europe-west1-c&quot;]}'
    cluster-autoscaler.kubernetes.io/safe-to-evict: &quot;true&quot;
    kubectl.kubernetes.io/last-applied-configuration: |
      {&quot;apiVersion&quot;:&quot;v1&quot;,&quot;kind&quot;:&quot;Service&quot;,&quot;metadata&quot;:{&quot;annotations&quot;:{&quot;cloud.google.com/app-protocols&quot;:&quot;{\&quot;web\&quot;:\&quot;HTTPS\&quot;}&quot;,&quot;cloud.google.com/backend-config&quot;:&quot;{\&quot;default\&quot;: \&quot;argo-server-backendconfig\&quot;}&quot;,&quot;cloud.google.com/neg&quot;:&quot;{\&quot;ingress\&quot;: true}&quot;,&quot;cluster-autoscaler.kubernetes.io/safe-to-evict&quot;:&quot;true&quot;},&quot;labels&quot;:{&quot;app.kubernetes.io/managed-by&quot;:&quot;gcp-cloud-build-deploy&quot;},&quot;name&quot;:&quot;argo-server&quot;,&quot;namespace&quot;:&quot;argo&quot;},&quot;spec&quot;:{&quot;ports&quot;:[{&quot;name&quot;:&quot;web&quot;,&quot;port&quot;:2746,&quot;targetPort&quot;:2746}],&quot;selector&quot;:{&quot;app&quot;:&quot;argo-server&quot;}}}
  creationTimestamp: &quot;2022-06-15T11:44:07Z&quot;
  labels:
    app.kubernetes.io/managed-by: gcp-cloud-build-deploy
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:cloud.google.com/app-protocols: {}
          f:cloud.google.com/backend-config: {}
          f:cluster-autoscaler.kubernetes.io/safe-to-evict: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/managed-by: {}
      f:spec:
        f:ports:
          .: {}
          k:{&quot;port&quot;:2746,&quot;protocol&quot;:&quot;TCP&quot;}:
            .: {}
            f:name: {}
            f:port: {}
            f:protocol: {}
            f:targetPort: {}
        f:selector:
          .: {}
          f:app: {}
        f:sessionAffinity: {}
        f:type: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: &quot;2022-06-15T12:27:23Z&quot;
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:cloud.google.com/neg: {}
    manager: GoogleGKEGatewayController
    operation: Update
    time: &quot;2022-06-15T12:28:06Z&quot;
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          f:cloud.google.com/neg-status: {}
    manager: glbc
    operation: Update
    time: &quot;2022-06-15T12:28:06Z&quot;
  name: argo-server
  namespace: argo
  resourceVersion: &quot;41692832&quot;
  uid: 25024d53-1d31-4165-8033-1843ec5d72ec
spec:
  clusterIP: 10.163.247.121
  clusterIPs:
  - 10.163.247.121
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: web
    port: 2746
    protocol: TCP
    targetPort: 2746
  selector:
    app: argo-server
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

</code></pre>

MBHA Phoenix
<p>I found a solution which I think of as a workaround.</p>
<ol>
<li><p>Use <code>networking.gke.io/app-protocols: '{&quot;web&quot;:&quot;HTTPS&quot;}'</code> annotation rather than  <code>cloud.google.com/app-protocols: '{&quot;web&quot;:&quot;HTTPS&quot;}'</code>. This annotation is to use at service level, where <code>web</code> is name of the port. This  will enable HTTPS between the load balancer and the application <em>(Endpoint protocol of the backend service created for the specified HTTPRoute)</em>. <strong>This is working perfectly with <code>gatewayClassName: gke-l7-rilb</code> a Regional Internal Load Balancer.</strong></p>
</li>
<li><p>create a custom health check using <code>cloud.google.com/v1 BackendConfig</code> where you set the type to HTTPS and port to 2746. More details here <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#direct_health" rel="nofollow noreferrer">https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#direct_health</a> <em>With ingresses, GCE ingress controller create automatically this health check from the application readiness probe but apparently this function is not yet implemented in the GKE Gateway controller.</em></p>
</li>
<li><p>Make sure you have firewall rules allowing ingress traffic for Google Cloud health checks on 2746 port <em>With ingresses, GCE ingress controller create automatically the required firewall rule but apparently this function is not yet implemented in the GKE Gateway controller.</em></p>
</li>
</ol>
<p>Finally I said this is a workaround because I imagine and hope that future version of the GKE gateway controller will fix the 3 issues or points I mentioned above.</p>
