<p>I'm having trouble to expose a k8s cluster deployed on AKS with a public IP address. I'm using GitHub Actions to do the deployment. The following are my .tf and deployment.yml files;</p>
<p>Please see below the errors I'm facing.</p>
<p>main.tf</p>
<pre><code>provider &quot;azurerm&quot; {
  features {}
}

provider &quot;azuread&quot; {
  version = &quot;=0.7.0&quot;
}

terraform {
  backend &quot;azurerm&quot; {
    resource_group_name  = &quot;tstate-rg&quot;
    storage_account_name = &quot;tstateidentity11223&quot;
    container_name       = &quot;tstate&quot;
    access_key           = &quot;/qSJCUo...&quot;
    key                  = &quot;terraform.tfstate&quot;
  }
}

# create resource group
resource &quot;azurerm_resource_group&quot; &quot;aks&quot; {
  name     = &quot;${var.name_prefix}-rg&quot;
  location = var.location
  }
}
</code></pre>
<p>aks-cluster.tf</p>
<pre><code>resource &quot;azurerm_kubernetes_cluster&quot; &quot;aks&quot; {
  name                = &quot;${var.name_prefix}-aks&quot;
  location            = var.location
  resource_group_name = var.resourcename
  dns_prefix          = &quot;${var.name_prefix}-dns&quot;

  default_node_pool {
    name            = &quot;identitynode&quot;
    node_count      = 3
    vm_size         = &quot;Standard_D2_v2&quot;
    os_disk_size_gb = 30
  }

  service_principal {
    client_id     = var.client_id
    client_secret = var.client_secret
  }

  network_profile {
    network_plugin    = &quot;kubenet&quot;
    load_balancer_sku = &quot;Standard&quot;
  }
}
</code></pre>
<p>nginxlb.tf</p>
<pre><code># Initialize Helm (and install Tiller)
provider &quot;helm&quot; {
  #  install_tiller = true

  kubernetes {
    host                   = azurerm_kubernetes_cluster.aks.kube_config.0.host
    client_certificate     = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate)
    client_key             = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_key)
    cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)
    load_config_file       = false
  }
}

# Add Kubernetes Stable Helm charts repo
data &quot;helm_repository&quot; &quot;stable&quot; {
  name = &quot;stable&quot;
  url  = &quot;https://kubernetes-charts.storage.googleapis.com&quot;
}

# Create Static Public IP Address to be used by Nginx Ingress
resource &quot;azurerm_public_ip&quot; &quot;nginx_ingress&quot; {
  name                = &quot;nginx-ingress-pip&quot;
  location            = azurerm_kubernetes_cluster.aks.location
  resource_group_name = azurerm_kubernetes_cluster.aks.node_resource_group

  allocation_method = &quot;Static&quot;
  domain_name_label = var.name_prefix
}


# Install Nginx Ingress using Helm Chart
resource &quot;helm_release&quot; &quot;nginx&quot; {
  name       = &quot;nginx-ingress&quot;
  repository = data.helm_repository.stable.url
  #repository = data.helm_repository.stable.metadata.0.name
  chart = &quot;nginx-ingress&quot;
  # namespace  = &quot;kube-system&quot;
  namespace = &quot;default&quot;

  set {
    name  = &quot;rbac.create&quot;
    value = &quot;false&quot;
  }

  set {
    name  = &quot;controller.service.externalTrafficPolicy&quot;
    value = &quot;Local&quot;
  }

  set {
    name  = &quot;controller.service.loadBalancerIP&quot;
    value = azurerm_public_ip.nginx_ingress.ip_address
  }
} 
</code></pre>
<p>And my deployment.yml</p>
<pre><code>apiVersion: v1
kind: Namespace
metadata:
  name: 
  namespace: default
---
apiVersion: v1
kind: Service
metadata:
  name: identity-svc
  namespace: default
  labels:
    name: identity-svc
    env: dev
    app: identity-svc
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-resource-group: MC_identity-k8s-rg_identity-k8s-aks_westeurope
    # nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  loadBalancerIP: 13.95.67.206
  type: LoadBalancer ## NodePort,ClusterIP,LoadBalancer --&gt; Ingress Controller:nginx,HAProxy
  ports:
  - name: http
    port: 8000
    targetPort: 8000
    nodePort: 30036
    protocol: TCP
  selector:
    app: identity-svc
---
apiVersion: v1
data:
  .dockerconfigjson: eyJhdXRocyI6eyJpZGVudGl0eXNlcnZpY2UuYXp1cmVjVZWcVpYS2o4QTM3RmsvZEZZbTlrbHQiLCJlbWFpbCI6InN1YmplQHN1YmplLmNvbSIsImF1dGgiOiJ
kind: Secret
metadata:
  creationTimestamp: null
  name: acr-secret
  namespace: default
type: kubernetes.io/dockerconfigjson
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: identity-deploy
  namespace: default
  labels:
    name: identity-app
    env: dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: identity-svc
  template:
    metadata:
      namespace: default
      labels:
        app: identity-svc
    spec:
      #backoffLimit: 1
      imagePullSecrets:
        - name: acr-secret
      containers:
      - name: identitysvc
        image: identitysvc.azurecr.io/identitysvc:${{ github.run_id }}
        env:
          - name: SECRET_KEY
            value: ${SECRET_KEY}
          - name: DOPPLER_TOKEN
            value: ${DOPPLER_TOKEN}
        resources:
          requests:
            cpu: 0.5
            memory: &quot;500Mi&quot; 
          limits:
            cpu: 2
            memory: &quot;1000Mi&quot;
        ports:
        - containerPort: 8000
          name: http
        imagePullPolicy: Always
      restartPolicy: Always
</code></pre>
<p>The following are the error messages from GitHub Actions log and Kubectl on Azure.</p>
<p>GitHub Actions log;
This message gets repeated until timeout.
<img src="https://i.stack.imgur.com/PgMNB.png" alt="" /></p>
<p>Kubectl logs on AKS;
<img src="https://i.stack.imgur.com/jhmdb.png" alt="" /></p>
<p>kubectl describe svc</p>
<pre><code>Name:                     nginx-ingress-controller
Namespace:                default
Labels:                   app=nginx-ingress
                          app.kubernetes.io/managed-by=Helm
                          chart=nginx-ingress-1.41.3
                          component=controller
                          heritage=Helm
                          release=nginx-ingress
Annotations:              meta.helm.sh/release-name: nginx-ingress
                          meta.helm.sh/release-namespace: default
Selector:                 app.kubernetes.io/component=controller,app=nginx-ingress,release=nginx-ingress
Type:                     LoadBalancer
IP:                       10.0.153.66
IP:                       13.95.67.206
Port:                     http  8000/TCP
TargetPort:               http/TCP
NodePort:                 http  30933/TCP
Endpoints:                10.244.1.6:8000
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  32230/TCP
Endpoints:                10.244.1.6:443
Session Affinity:         None
External Traffic Policy:  Local
HealthCheck NodePort:     32755
Events:
  Type     Reason                      Age                     From                  Message
  ----     ------                      ----                    ----                  -------
  Normal   EnsuringLoadBalancer        4m17s (x43 over 3h10m)  service-controller    Ensuring load balancer
  Warning  CreateOrUpdateLoadBalancer  4m16s (x43 over 3h10m)  azure-cloud-provider  Code=&quot;PublicIPAndLBSkuDoNotMatch&quot; Message=&quot;Standard sku load balancer /subscriptions/e90bd4d0-3b50-4a27-a7e8-bc88cf5f5398/resourceGroups/mc_identity-k8s-rg_identity-k8s-aks_westeurope/providers/Microsoft.Network/loadBalancers/kubernetes cannot reference Basic sku publicIP /subscriptions/e90bd4d0-3b50-4a27-a7e8-bc88cf5f5398/resourceGroups/MC_identity-k8s-rg_identity-k8s-aks_westeurope/providers/Microsoft.Network/publicIPAddresses/nginx-ingress-pip.&quot; Details=[]
</code></pre>
<p>kubectl logs</p>
<pre><code>I1108 12:52:52.862797       7 flags.go:205] Watching for Ingress class: nginx
W1108 12:52:52.863034       7 flags.go:250] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W1108 12:52:52.863078       7 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1108 12:52:52.863272       7 main.go:231] Creating API client for https://10.0.0.1:443
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v0.34.1
  Build:         v20200715-ingress-nginx-2.11.0-8-gda5fa45e2
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.1

-------------------------------------------------------------------------------

I1108 12:52:52.892455       7 main.go:275] Running in Kubernetes cluster version v1.17 (v1.17.13) - git (clean) commit 30d651da517185653e34e7ab99a792be6a3d9495 - platform linux/amd64
I1108 12:52:52.897887       7 main.go:87] Validated default/nginx-ingress-default-backend as the default backend.
I1108 12:52:53.229870       7 main.go:105] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
W1108 12:52:53.252657       7 store.go:659] Unexpected error reading configuration configmap: configmaps &quot;nginx-ingress-controller&quot; not found
I1108 12:52:53.268067       7 nginx.go:263] Starting NGINX Ingress controller
I1108 12:52:54.468656       7 leaderelection.go:242] attempting to acquire leader lease  default/ingress-controller-leader-nginx...
I1108 12:52:54.468691       7 nginx.go:307] Starting NGINX process
W1108 12:52:54.469222       7 controller.go:395] Service &quot;default/nginx-ingress-default-backend&quot; does not have any active Endpoint
I1108 12:52:54.469249       7 controller.go:141] Configuration changes detected, backend reload required.
I1108 12:52:54.473464       7 status.go:86] new leader elected: nginx-ingress-controller-6b45fcd8ff-7mbx4
I1108 12:52:54.543113       7 controller.go:157] Backend successfully reloaded.
I1108 12:52:54.543152       7 controller.go:166] Initial sync, sleeping for 1 second.
W1108 12:52:58.251867       7 controller.go:395] Service &quot;default/nginx-ingress-default-backend&quot; does not have any active Endpoint
I1108 12:53:38.008002       7 leaderelection.go:252] successfully acquired lease default/ingress-controller-leader-nginx
I1108 12:53:38.008203       7 status.go:86] new leader elected: nginx-ingress-controller-6b45fcd8ff-njgjs
</code></pre>
<p>Help me understand what am I missing here? This whole process is me trying to deploy a simple Python service on a public IP address. I'm just trying to expose the service on a public IP it doesn't matter currently with which method whether nginx or any other loadbalancing service.</p>
<p>Also before I implement nginx ingress in the Terraform files I could see identity-svc running when I ran <code>kubectl get services</code> but now I can't even see that service its only the nginx ingress controller. I really appreciate any help.</p>
<p><strong>Edit:</strong> After adding sku standard to the public IP creation as @mynko mentioned the workflow could run successfully. Now when I check the following;</p>
<pre><code>admin@Azure:~$ kubectl get svc
NAME                            TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
identity-svc                    LoadBalancer   10.0.188.32    20.56.242.212   8000:30036/TCP               22m
kubernetes                      ClusterIP      10.0.0.1       &lt;none&gt;          443/TCP                      7h28m
nginx-ingress-controller        LoadBalancer   10.0.230.164   20.50.221.84    8000:31742/TCP,443:31675/TCP   23m
nginx-ingress-default-backend   ClusterIP      10.0.229.217   &lt;none&gt;          8000/TCP                       23m
</code></pre>
<p>I get this I'm not sure why <code>nginx-ingress-controller</code> is looking at port 80 instead of 8000. Also when I try to access `20.56.242.212:8000 nothing loads up. Also in this case which one should be my exposed public IP?</p>
<p>When I access <code>20.50.221.84</code> it shows <code>default backend - 404</code></p>

Mert Alnuaimi
<p>Have a look at the kubernetes service warning message.</p>
<pre><code>Code=&quot;PublicIPAndLBSkuDoNotMatch&quot;
</code></pre>
<p>You're using a Basic SKU Public IP, change it to Standard.</p>
<p><a href="https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/public_ip#sku" rel="noreferrer">https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/public_ip#sku</a></p>
