<p>My goal is do deploy a self-hosted instance of GitLab on <a href="https://aws.amazon.com/eks/" rel="nofollow noreferrer">EKS</a>. I have read through the <a href="https://docs.gitlab.com/charts/installation/operator.html" rel="nofollow noreferrer">guide</a> on the GitLab docs and am attempting the operator install method. I setup my cluster using <a href="https://eksctl.io/" rel="nofollow noreferrer">eksctl</a> v0.61.0 and three t4g.large instances. The cluster comes up and seems healthy.</p>
<pre class="lang-sh prettyprint-override"><code>kubectl get all --all-namespaces
</code></pre>
<pre class="lang-sh prettyprint-override"><code>NAMESPACE     NAME                           READY   STATUS    RESTARTS   AGE
kube-system   pod/aws-node-9k7mg             1/1     Running   0          3m25s
kube-system   pod/aws-node-hlkxr             1/1     Running   0          3m25s
kube-system   pod/aws-node-rc5br             1/1     Running   0          3m24s
kube-system   pod/coredns-5c778788f4-cw5gq   1/1     Running   0          15m
kube-system   pod/coredns-5c778788f4-ff8mn   1/1     Running   0          15m
kube-system   pod/kube-proxy-hrxtz           1/1     Running   0          3m25s
kube-system   pod/kube-proxy-phw7p           1/1     Running   0          3m25s
kube-system   pod/kube-proxy-rtlgj           1/1     Running   0          3m25s

NAMESPACE     NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)         AGE
default       service/kubernetes   ClusterIP   10.100.0.1    &lt;none&gt;        443/TCP         16m
kube-system   service/kube-dns     ClusterIP   10.100.0.10   &lt;none&gt;        53/UDP,53/TCP   16m

NAMESPACE     NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   daemonset.apps/aws-node     3         3         3       3            3           &lt;none&gt;          16m
kube-system   daemonset.apps/kube-proxy   3         3         3       3            3           &lt;none&gt;          16m

NAMESPACE     NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   deployment.apps/coredns   2/2     2            2           16m

NAMESPACE     NAME                                 DESIRED   CURRENT   READY   AGE
kube-system   replicaset.apps/coredns-5c778788f4   2         2         2       15m
</code></pre>
<p>I begin by installing <a href="https://cert-manager.io/docs/installation/" rel="nofollow noreferrer">cert-manager</a> v1.6.0 using the default config.</p>
<pre class="lang-sh prettyprint-override"><code>kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.6.0/cert-manager.yaml
kubectl get all -n cert-manager
</code></pre>
<pre class="lang-sh prettyprint-override"><code>NAME                                           READY   STATUS    RESTARTS   AGE
pod/cert-manager-77fd97f598-wxtj8              1/1     Running   0          18s
pod/cert-manager-cainjector-7974c84449-ghlfr   1/1     Running   0          18s
pod/cert-manager-webhook-5f4b965fbd-8kqv2      1/1     Running   0          17s

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/cert-manager           ClusterIP   10.100.71.170    &lt;none&gt;        9402/TCP   18s
service/cert-manager-webhook   ClusterIP   10.100.191.224   &lt;none&gt;        443/TCP    18s

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cert-manager              1/1     1            1           19s
deployment.apps/cert-manager-cainjector   1/1     1            1           19s
deployment.apps/cert-manager-webhook      1/1     1            1           18s

NAME                                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/cert-manager-77fd97f598              1         1         1       19s
replicaset.apps/cert-manager-cainjector-7974c84449   1         1         1       19s
replicaset.apps/cert-manager-webhook-5f4b965fbd      1         1         1       18s
</code></pre>
<p>Next, I install the metrics server</p>
<pre class="lang-sh prettyprint-override"><code>kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
</code></pre>
<p>This comes up as well and does not appear to have any issues.</p>
<p>Finally, I attempt to install the GitLab operator</p>
<pre class="lang-sh prettyprint-override"><code>GL_OPERATOR_VERSION=0.1.0
PLATFORM=kubernetes
kubectl create namespace gitlab-system
kubectl apply -f https://gitlab.com/api/v4/projects/18899486/packages/generic/gitlab-operator/${GL_OPERATOR_VERSION}/gitlab-operator-${PLATFORM}-${GL_OPERATOR_VERSION}.yaml
</code></pre>
<p>*Note: The newest release of cert-manager at the time of this post was <a href="https://github.com/jetstack/cert-manager/releases" rel="nofollow noreferrer">1.6.0</a>. During this update, the APIVersions <code>v1alpha2, v1alpha3, and v1beta1</code> were deprecated. When I attempted this install it failed to create the Issuer and Certificate. Updating the APIVersions to <code>cert-manager.io/v1</code> fixed this.</p>
<p>Now, it creates all the resources.</p>
<pre class="lang-sh prettyprint-override"><code>kubectl get all -n gitlab-system
</code></pre>
<pre class="lang-sh prettyprint-override"><code>NAME                                            READY   STATUS             RESTARTS   AGE
pod/gitlab-controller-manager-ccd797cb6-9c428   0/2     CrashLoopBackOff   4          30s

NAME                                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/gitlab-controller-manager-metrics-service   ClusterIP   10.100.252.76   &lt;none&gt;        8443/TCP   30s
service/gitlab-webhook-service                      ClusterIP   10.100.85.217   &lt;none&gt;        443/TCP    30s

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/gitlab-controller-manager   0/1     1            0           30s

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/gitlab-controller-manager-ccd797cb6   1         1         0       30s
</code></pre>
<p>As shown, the <code>pod/gitlab-controller-manager-ccd797cb6-9c428</code> is in a <code>CrashLoopBackOff</code> state. It continues to restart indefinitely.</p>
<pre class="lang-sh prettyprint-override"><code>kubectl describe pod gitlab-controller-manager-ccd797cb6-9c428 -n gitlab-system
</code></pre>
<pre class="lang-sh prettyprint-override"><code>Name:         gitlab-controller-manager-ccd797cb6-9c428
Namespace:    gitlab-system
Priority:     0
Node:         ip-192-168-78-2.us-east-2.compute.internal/192.168.78.2
Start Time:   Thu, 28 Oct 2021 18:13:28 -0400
Labels:       control-plane=controller-manager
              pod-template-hash=ccd797cb6
Annotations:  kubernetes.io/psp: eks.privileged
Status:       Running
IP:           192.168.95.73
IPs:
  IP:           192.168.95.73
Controlled By:  ReplicaSet/gitlab-controller-manager-ccd797cb6
Containers:
  manager:
    Container ID:  docker://8576f635b72389a824284a1c342c390036af50bf85a60aa3299af17d77764971
    Image:         registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0
    Image ID:      docker-pullable://registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator@sha256:3d0ff0fc176511d67f3784060023157fbdaed8109539f3d340d68ac8f18d6425
    Ports:         9443/TCP, 6060/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /manager
    Args:
      --metrics-addr=127.0.0.1:8080
      --enable-leader-election
      --zap-devel=true
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 28 Oct 2021 18:14:24 -0400
      Finished:     Thu, 28 Oct 2021 18:14:24 -0400
    Ready:          False
    Restart Count:  3
    Limits:
      cpu:     200m
      memory:  300Mi
    Requests:
      cpu:      200m
      memory:   100Mi
    Liveness:   http-get http://:health-port/liveness delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:  http-get http://:health-port/readiness delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment:
      WATCH_NAMESPACE:  gitlab-system (v1:metadata.namespace)
    Mounts:
      /tmp/k8s-webhook-server/serving-certs from cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from gitlab-manager-token-vjdfx (ro)
  kube-rbac-proxy:
    Container ID:  docker://1db8028b18e0e7f255f1fdc1c0ab086d0cb01d17a10e3b0d17b9a8e6afda9175
    Image:         gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
    Image ID:      docker-pullable://gcr.io/kubebuilder/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b
    Port:          8443/TCP
    Host Port:     0/TCP
    Args:
      --secure-listen-address=0.0.0.0:8443
      --upstream=http://127.0.0.1:8080/
      --logtostderr=true
      --v=10
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 28 Oct 2021 18:14:24 -0400
      Finished:     Thu, 28 Oct 2021 18:14:24 -0400
    Ready:          False
    Restart Count:  3
    Environment:    &lt;none&gt;
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from gitlab-manager-token-vjdfx (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  webhook-server-cert
    Optional:    false
  gitlab-manager-token-vjdfx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  gitlab-manager-token-vjdfx
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason       Age                From               Message
  ----     ------       ----               ----               -------
  Normal   Scheduled    61s                default-scheduler  Successfully assigned gitlab-system/gitlab-controller-manager-ccd797cb6-9c428 to ip-192-168-78-2.us-east-2.compute.internal
  Warning  FailedMount  60s (x2 over 61s)  kubelet            MountVolume.SetUp failed for volume &quot;cert&quot; : secret &quot;webhook-server-cert&quot; not found
  Normal   Pulling      55s                kubelet            Pulling image &quot;gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0&quot;
  Normal   Pulled       55s                kubelet            Successfully pulled image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot; in 3.560963186s
  Normal   Pulled       53s                kubelet            Successfully pulled image &quot;gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0&quot; in 1.650875485s
  Normal   Pulled       52s                kubelet            Container image &quot;gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0&quot; already present on machine
  Normal   Created      52s (x2 over 53s)  kubelet            Created container kube-rbac-proxy
  Normal   Started      52s (x2 over 53s)  kubelet            Started container kube-rbac-proxy
  Normal   Pulled       52s                kubelet            Successfully pulled image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot; in 490.074654ms
  Warning  BackOff      50s (x2 over 51s)  kubelet            Back-off restarting failed container
  Warning  BackOff      50s (x2 over 51s)  kubelet            Back-off restarting failed container
  Normal   Pulling      39s (x3 over 59s)  kubelet            Pulling image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot;
  Normal   Started      38s (x3 over 55s)  kubelet            Started container manager
  Normal   Created      38s (x3 over 55s)  kubelet            Created container manager
  Normal   Pulled       38s                kubelet            Successfully pulled image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot; in 512.734325ms
</code></pre>
<p>The only issue I recognize is the missing &quot;webhook-server-cert&quot; secret.</p>
<pre class="lang-sh prettyprint-override"><code>kubectl get secrets -n gitlab-system
</code></pre>
<pre class="lang-sh prettyprint-override"><code>NAME                               TYPE                                  DATA   AGE
default-token-tzxs2                kubernetes.io/service-account-token   3      86s
gitlab-app-token-7btgp             kubernetes.io/service-account-token   3      83s
gitlab-manager-token-vjdfx         kubernetes.io/service-account-token   3      83s
gitlab-nginx-ingress-token-v5jdh   kubernetes.io/service-account-token   3      82s
webhook-server-cert                kubernetes.io/tls                     3      80s
</code></pre>
<p>The secret is there and when I run <code>get</code> on it I can see the cert and key.</p>
<p>Here is the result of running <code>kubectl get events -n gitlab-system</code></p>
<pre class="lang-sh prettyprint-override"><code>LAST SEEN   TYPE      REASON              OBJECT                                           MESSAGE
100s        Normal    Scheduled           pod/gitlab-controller-manager-ccd797cb6-9c428    Successfully assigned gitlab-system/gitlab-controller-manager-ccd797cb6-9c428 to ip-192-168-78-2.us-east-2.compute.internal
99s         Warning   FailedMount         pod/gitlab-controller-manager-ccd797cb6-9c428    MountVolume.SetUp failed for volume &quot;cert&quot; : secret &quot;webhook-server-cert&quot; not found
78s         Normal    Pulling             pod/gitlab-controller-manager-ccd797cb6-9c428    Pulling image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot;
94s         Normal    Pulled              pod/gitlab-controller-manager-ccd797cb6-9c428    Successfully pulled image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot; in 3.560963186s
77s         Normal    Created             pod/gitlab-controller-manager-ccd797cb6-9c428    Created container manager
77s         Normal    Started             pod/gitlab-controller-manager-ccd797cb6-9c428    Started container manager
94s         Normal    Pulling             pod/gitlab-controller-manager-ccd797cb6-9c428    Pulling image &quot;gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0&quot;
92s         Normal    Pulled              pod/gitlab-controller-manager-ccd797cb6-9c428    Successfully pulled image &quot;gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0&quot; in 1.650875485s
91s         Normal    Created             pod/gitlab-controller-manager-ccd797cb6-9c428    Created container kube-rbac-proxy
91s         Normal    Started             pod/gitlab-controller-manager-ccd797cb6-9c428    Started container kube-rbac-proxy
91s         Normal    Pulled              pod/gitlab-controller-manager-ccd797cb6-9c428    Successfully pulled image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot; in 490.074654ms
91s         Normal    Pulled              pod/gitlab-controller-manager-ccd797cb6-9c428    Container image &quot;gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0&quot; already present on machine
89s         Warning   BackOff             pod/gitlab-controller-manager-ccd797cb6-9c428    Back-off restarting failed container
89s         Warning   BackOff             pod/gitlab-controller-manager-ccd797cb6-9c428    Back-off restarting failed container
77s         Normal    Pulled              pod/gitlab-controller-manager-ccd797cb6-9c428    Successfully pulled image &quot;registry.gitlab.com/gitlab-org/cloud-native/gitlab-operator:0.1.0&quot; in 512.734325ms
100s        Normal    SuccessfulCreate    replicaset/gitlab-controller-manager-ccd797cb6   Created pod: gitlab-controller-manager-ccd797cb6-9c428
100s        Normal    ScalingReplicaSet   deployment/gitlab-controller-manager             Scaled up replica set gitlab-controller-manager-ccd797cb6 to 1
99s         Normal    cert-manager.io     certificaterequest/gitlab-serving-cert-ghlz8     Certificate request has been approved by cert-manager.io
99s         Warning   BadConfig           certificaterequest/gitlab-serving-cert-ghlz8     Certificate will be issued with an empty Issuer DN, which contravenes RFC 5280 and could break some strict clients
99s         Normal    CertificateIssued   certificaterequest/gitlab-serving-cert-ghlz8     Certificate fetched from issuer successfully
99s         Normal    Issuing             certificate/gitlab-serving-cert                  Issuing certificate as Secret does not exist
99s         Normal    Generated           certificate/gitlab-serving-cert                  Stored new private key in temporary Secret resource &quot;gitlab-serving-cert-k5djd&quot;
99s         Normal    Requested           certificate/gitlab-serving-cert                  Created new CertificateRequest resource &quot;gitlab-serving-cert-ghlz8&quot;
99s         Normal    Issuing             certificate/gitlab-serving-cert                  The certificate has been successfully issued
</code></pre>
<p>I'm not sure how to approach this issue. Any insights?</p>

Seth Faulkner
<p>After investigating a little more, I found that running logs on the container yielded <code>standard_init_linux.go:228: exec user process caused: exec format error</code> I opened an issue with the GitLab Operator project and they advised that GitLab Operator must be run on x64_86 architecture. The T4g series is AArch64/arm64. I switched to t2.xlarge and was able to bring up the operator.</p>
