This is an example of how to create both an internal and an external ingressClass on an azure AKS cluster, consumable by specifying the ingressClass name in ingress resources.

The key is the azure-load-balancer-internal annotation on the loadbalancer fronting the ingress controllers. This AKS cluster is using the Azure CNI.

Of course we’re using declarative helm + terraform for true future accountability when these ingresses need upgraded / reconfigured.

resource "helm_release" "nginx-internal" {
  name      = "internal"
  namespace = "default"

  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"

  set {
    name  = "controller.ingressClassResource.name"
    value = "internal"
  }
  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-internal"
    value = "true"
  }
  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-health-probe-request-path"
    value = "/healthz"
  }
}

resource "helm_release" "nginx-external" {
  name      = "external"
  namespace = "default"

  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"

  set {
    name  = "controller.ingressClassResource.name"
    value = "external"
  }
  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-health-probe-request-path"
    value = "/healthz"
  }
}

> kubectl get ingressclass
NAME       CONTROLLER             PARAMETERS   AGE
external   k8s.io/ingress-nginx   <none>       3m57s
internal   k8s.io/ingress-nginx   <none>       15d

> kubectl get pods | grep nginx
external-ingress-nginx-controller-6c75485cb5-b2kwb      1/1     Running   0               68s
internal-ingress-nginx-controller-6c6c6cc767-24cgw      1/1     Running   0               8d

The result is a public ip on the external LoadBalancer, and a private ip from our cluster VNET on the internal LoadBalancer.

> kubectl get services | grep nginx
external-ingress-nginx-controller             LoadBalancer   172.29.243.108   1.2.3.4           80:31642/TCP,443:30547/TCP   90s
external-ingress-nginx-controller-admission   ClusterIP      172.29.191.186   <none>            443/TCP                      90s
internal-ingress-nginx-controller             LoadBalancer   172.29.169.30    10.10.10.134      80:31003/TCP,443:32680/TCP   15d
internal-ingress-nginx-controller-admission   ClusterIP      172.29.134.99    <none>            443/TCP                      15d

The ingress of choice can then be selected as such:

resource "kubernetes_ingress_v1" "nginx" {
  metadata {
    name = "nginx"
  }
  spec {
    ingress_class_name = "external"
    rule {
      host = var.fqdn
      http {
        path {
          backend {
            service {
              name = "nginx"
              port {
                number = 80
              }
            }
          }
          path      = "/app"
          path_type = "Prefix"
        }
      }
    }
    tls {
      hosts       = [var.fqdn]
      secret_name = var.cert_secret
    }
  }
}