<p>I was able to successfully start <strong>keycloak server</strong> on <strong>AWS K3S Kubernetes Cluster</strong> with <strong>Istio Gateway</strong> and <strong>AWS HTTPS Application Load Balancer</strong>.</p>
<p>I can successfully see <strong>Keycloak Home Page</strong>:
<strong><a href="https://keycloak.skycomposer.net/auth/" rel="nofollow noreferrer">https://keycloak.skycomposer.net/auth/</a></strong></p>
<p>But when I click on <strong>Admin Console</strong> link, then the <strong>Blank Page</strong> is shown:
<strong><a href="https://keycloak.skycomposer.net/auth/admin/master/console/" rel="nofollow noreferrer">https://keycloak.skycomposer.net/auth/admin/master/console/</a></strong></p>
<p><strong>Browser Inspect Tool</strong> shows that:
<strong><a href="http://keycloak.skycomposer.net/auth/js/keycloak.js?version=rk826" rel="nofollow noreferrer">http://keycloak.skycomposer.net/auth/js/keycloak.js?version=rk826</a></strong>
link returns the following status:</p>
<pre><code>(blocked:mixed-content)
</code></pre>
<p>I did some research on the internet and the reason seems to be related with redirection from <strong>https</strong> to <strong>http</strong>, which is not correctly handled by <strong>istio gateway</strong> and <strong>aws load balancer</strong></p>
<p>But unfortunately, I couldn't find the solution, how to solve it for my particular environment.</p>
<p>Here are my configuration files:</p>
<p><strong>keycloak-config.yaml:</strong></p>
<pre><code>apiVersion: v1
kind: ConfigMap
metadata:
  name: keycloak
data:
  KEYCLOAK_USER: admin@keycloak
  KEYCLOAK_MGMT_USER: mgmt@keycloak
  JAVA_OPTS_APPEND: '-Djboss.http.port=8080'
  PROXY_ADDRESS_FORWARDING: 'true'
  KEYCLOAK_HOSTNAME: 'keycloak.skycomposer.net'
  KEYCLOAK_FRONTEND_URL: 'https://keycloak.skycomposer.net/auth'  
  KEYCLOAK_LOGLEVEL: INFO
  ROOT_LOGLEVEL: INFO
  DB_VENDOR: H2
</code></pre>
<p><strong>keycloak-deployment.yaml:</strong></p>
<pre><code>kind: Deployment
apiVersion: apps/v1
metadata:
  name: keycloak
  labels:
    app: keycloak

spec:
  replicas: 1
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
      annotations:
        sidecar.istio.io/rewriteAppHTTPProbers: &quot;true&quot;
    spec:
      containers:
        - name: keycloak
          image: jboss/keycloak:13.0.1
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
              hostPort: 8080
          volumeMounts:
            - name: keycloak-data
              mountPath: /opt/jboss/keycloak/standalone/data
          env:
            - name: KEYCLOAK_USER
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: KEYCLOAK_USER
            - name: KEYCLOAK_MGMT_USER
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: KEYCLOAK_MGMT_USER
            - name: JAVA_OPTS_APPEND
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: JAVA_OPTS_APPEND
            - name: DB_VENDOR
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: DB_VENDOR
            - name: PROXY_ADDRESS_FORWARDING
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: PROXY_ADDRESS_FORWARDING
            - name: KEYCLOAK_HOSTNAME
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: KEYCLOAK_HOSTNAME
            - name: KEYCLOAK_FRONTEND_URL
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: KEYCLOAK_FRONTEND_URL
            - name: KEYCLOAK_LOGLEVEL
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: KEYCLOAK_LOGLEVEL
            - name: ROOT_LOGLEVEL
              valueFrom:
                configMapKeyRef:
                  name: keycloak
                  key: ROOT_LOGLEVEL
            - name: KEYCLOAK_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: keycloak
                  key: KEYCLOAK_PASSWORD
            - name: KEYCLOAK_MGMT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: keycloak
                  key: KEYCLOAK_MGMT_PASSWORD
      volumes:
        - name: keycloak-data
          persistentVolumeClaim:
            claimName: keycloak-pvc
</code></pre>
<p><strong>keycloak-service.yaml:</strong></p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: keycloak

spec:
  ports:
    - protocol: TCP
      name: http
      port: 80
      targetPort: 8080
  selector:
    app: keycloak
</code></pre>
<p><strong>istio-gateway.yaml:</strong></p>
<pre><code>apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: istio-gateway
spec: 
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - &quot;keycloak.skycomposer.net&quot;
</code></pre>
<p><strong>istio-virtualservice.yaml:</strong></p>
<pre><code>apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: keycloak
spec:
  hosts:
    - keycloak.skycomposer.net
  gateways:
    - istio-gateway
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: keycloak.default.svc.cluster.local
            port:
              number: 80
</code></pre>
<p>I successfully installed <strong>istio 1.9.1</strong> with <strong>istioctl</strong>:</p>
<pre><code>istioctl install \
  --set meshConfig.accessLogFile=/dev/stdout \
  --skip-confirmation
</code></pre>
<p>Also, I labelled default namespace with <strong>istio injection</strong>, so all my pods in default namespace have <strong>istio sidecar container</strong>:</p>
<pre><code>kubectl label namespace default istio-injection=enabled

NAME                                         READY   STATUS    RESTARTS   AGE
whoami-6c4757bbb5-9zkbl                      2/2     Running   0          13m
notification-microservice-5dfcf96b95-ll8lm   2/2     Running   0          13m
customermgmt-6b48586868-ddlnw                2/2     Running   0          13m
usermgmt-c5b65964-df2vc                      2/2     Running   0          13m
keycloak-d48f9bbbf-tsm5h                     2/2     Running   0          13m
</code></pre>
<p>Here is also <strong>terraform</strong> configuration of <strong>AWS Load Balancer</strong>:</p>
<pre><code>resource &quot;aws_lb&quot; &quot;mtc_lb&quot; {
  name            = &quot;mtc-loadbalancer&quot;
  subnets         = var.public_subnets
  security_groups = [var.public_sg]
  idle_timeout    = 400
}

resource &quot;aws_lb_target_group&quot; &quot;mtc_tg&quot; {
  name     = &quot;mtc-lb-tg-${substr(uuid(), 0, 3)}&quot;
  port     = var.tg_port
  protocol = var.tg_protocol
  vpc_id   = var.vpc_id
  lifecycle {
    create_before_destroy = true
    ignore_changes        = [name]
  }
  health_check {
    healthy_threshold   = var.elb_healthy_threshold
    unhealthy_threshold = var.elb_unhealthy_threshold
    timeout             = var.elb_timeout
    interval            = var.elb_interval
  }
}

resource &quot;aws_lb_listener&quot; &quot;mtc_lb_listener_http&quot; {
  load_balancer_arn = aws_lb.mtc_lb.arn
  port              = 80
  protocol          = &quot;HTTP&quot;

  default_action {
    type = &quot;redirect&quot;

    redirect {
      port        = &quot;443&quot;
      protocol    = &quot;HTTPS&quot;
      status_code = &quot;HTTP_301&quot;
    }
  }
}

resource &quot;aws_lb_listener&quot; &quot;mtc_lb_listener&quot; {
  load_balancer_arn = aws_lb.mtc_lb.arn
  port              = 443
  protocol          = &quot;HTTPS&quot;
  depends_on        = [aws_lb_target_group.mtc_tg]
  certificate_arn   = var.certificate_arn

  default_action {
    type             = &quot;forward&quot;
    target_group_arn = aws_lb_target_group.mtc_tg.arn
  }
}
</code></pre>

<p>Investigating <strong>request headers</strong>, I finally found the cause of the issue.</p>
<p>This header was always &quot;<strong>http</strong>&quot; by default:</p>
<pre><code>X-Forwarded-Proto: http
</code></pre>
<p>Changing the value to:</p>
<pre><code>X-Forwarded-Proto: https
</code></pre>
<p>solved the issue.</p>
<p>Here is the example of <strong>Istio Virtual Service</strong>, which sets &quot;<strong>X-Forwarded-Proto</strong>&quot; request header to &quot;<strong>https</strong>&quot; for all requests:</p>
<pre><code>apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: keycloak
spec:
  hosts:
    - keycloak.skycomposer.net
  gateways:
    - istio-gateway
  http:
    - match:
        - uri:
            prefix: /
      route:
        - destination:
            host: keycloak.default.svc.cluster.local
            port:
              number: 80
          headers:
            request:
              set:
                x-forwarded-proto: https
</code></pre>
<p>P.S. Ideal solution would be to set this value in <strong>AWS Application Load Balancer</strong>, but I wasn't sure how to do it with my <strong>terraform configuration</strong> of <strong>aws load balancer</strong>, so I decided to solve it on <strong>Istio Virtual Service</strong> level.</p>
