Sunday, April 26, 2020

Test drive Azure Application Gateway ingress controller in Azure Kubernetes Service (AKS)

Architecture Diagram


Azure Application Gateway ingress Controller in AKS


Overview

This blog is inspired by one of my recent partner engagements. There’s a fantastic article here about this new Azure solution announcement.  Let's understand the need for ingress controllers first.  In my previous post, we took a closer look at Kubenet networking and services in AKS. When you create a Service of the type LoadBalancer, an underlying load balancer resource is created in Azure. The load balancer is configured to distribute traffic to the pods on a given port. This Load Balancer works at layer 4 and the service is unaware of the higher layer applications (http/https).  Ingress controller works at layer 7 and can use more intelligent rules to distribute application traffic and also conserver public IP space.  The common use case is to route traffic to different applications based on the inbound URL.


In this post we drill a little deeper into the architecture, take a closer look at two types of Ingress services – Simple Fanout and Name Based Virtual hosting and take it for a test drive. Application gateway as ingress controller has direct access to all kubernetes pods and performs the layer 7 functions. In the previous blog I covered the in-cluster ingress controller with nginx and AKS.



Test Drive 

Follow the link to test drive application gateway ingress controller in Azure Kubernetes service.  Create your own sample deployments using the sample yaml manifests.

Fanout

http://colors.penguintrails.com
http://colors.penguintrails.com/red
http://colors.penguintrails.com/green
http://colors.penguintrails.com/blue
http://colors.penguintrails.com/white

Virtual Host

http://red.penguintrails.com
http://green.penguintrails.com
http://blue.penguintrails.com
default-white



Github

Pre-requisites:



  1. A Working AKS cluster with Application Gateway. Detailed documentation here.
  2.  Link to my github repo here. Git clone my repository and off you go..


#
# Create Namespace
kubectl create namespace colors-ns
#
# Deployments
#
kubectl apply -f red-deployment.yaml
kubectl apply -f green-deployment.yaml
kubectl apply -f blue-deployment.yaml
kubectl apply -f white-deployment.yaml
#
# Ingress
#
kubectl apply -f colors-fanout.yaml
kubectl apply -f colors-virtual-host.yaml

Pre-requisites

From Azure Documentation here

Ingress Concepts

Some ingress concepts here

Simple fanout


A fanout configuration routes traffic from a single IP address to more than one Service, based on the HTTP URI being requested. An Ingress allows you to keep the number of load balancers down to a minimum. For example, a setup like:

foo.bar.com -> 178.91.123.132 -> / foo    service1:4200
                                                       / bar    service2:8080

Name based virtual hosting


Name-based virtual hosts support routing HTTP traffic to multiple host names at the same IP address.

foo.bar.com --|                             |-> foo.bar.com service1:80
                       | 178.91.123.132  |
bar.foo.com --|                             |-> bar.foo.com service2:80



Validations

Lets follow the red-service all the way.

Fanout 


kubectl describe ingress colors-fanout-ingress -n colors-ns
Name:             colors-fanout-ingress
Namespace:        colors-ns
Address:          52.149.191.58
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                      Path  Backends
  ----                      ----  --------
  colors.penguintrails.com
                            /red     red-service:8080 (15.0.0.23:8080,15.0.0.50:8080,15.0.0.79:8080)
                            /green   green-service:8080 (15.0.0.18:8080,15.0.0.46:8080,15.0.0.92:8080)
                            /blue    blue-service:8080 (15.0.0.5:8080,15.0.0.65:8080,15.0.0.78:8080)
                                     white-service:8080 (15.0.0.15:8080,15.0.0.61:8080,15.0.0.90:8080)
Annotations:
  <snip>


Virtual Host

kubectl describe ingress colors-virtual-host-ingress -n colors-ns
Name:             colors-virtual-host-ingress
Namespace:        colors-ns
Address:          52.149.191.58
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                     Path  Backends
  ----                     ----  --------
  red.penguintrails.com
                              red-service:8080 (15.0.0.23:8080,15.0.0.50:8080,15.0.0.79:8080)
  green.penguintrails.com
                              green-service:8080 (15.0.0.18:8080,15.0.0.46:8080,15.0.0.92:8080)
  blue.penguintrails.com
                              blue-service:8080 (15.0.0.5:8080,15.0.0.65:8080,15.0.0.78:8080)
  *
                              white-service:8080 (15.0.0.15:8080,15.0.0.61:8080,15.0.0.90:8080)
Annotations:
 <snip>





AKS Nodes

kubectl get nodes -o wide
NAME                                STATUS   ROLES   AGE   VERSION    INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-agentpool-41427971-vmss000000   Ready    agent   9d    v1.15.10   15.0.0.4      <none>        Ubuntu 16.04.6 LTS   4.15.0-1071-azure   docker://3.0.10+azure
aks-agentpool-41427971-vmss000001   Ready    agent   9d    v1.15.10   15.0.0.35     <none>        Ubuntu 16.04.6 LTS   4.15.0-1071-azure   docker://3.0.10+azure
aks-agentpool-41427971-vmss000002   Ready    agent   9d    v1.15.10   15.0.0.66     <none>        Ubuntu 16.04.6 LTS   4.15.0-1071-azure   docker://3.0.10+azure

AKS Pods and Service 


kubectl get pods,service,ingress -o wide -n colors-ns                                                            

kubectl get pods,service,ingress -o wide -n colors-ns
NAME                                    READY   STATUS    RESTARTS   AGE    IP          NODE                                NOMINATED NODE   READINESS GATES
pod/blue-deployment-69d4cf87cd-6bz5p    1/1     Running   0          106m   15.0.0.65   aks-agentpool-41427971-vmss000001   <none>           <none>
pod/blue-deployment-69d4cf87cd-sx5l9    1/1     Running   0          106m   15.0.0.78   aks-agentpool-41427971-vmss000002   <none>           <none>
pod/blue-deployment-69d4cf87cd-vksvv    1/1     Running   0          106m   15.0.0.5    aks-agentpool-41427971-vmss000000   <none>           <none>
pod/green-deployment-5dbccd4bb9-25fk6   1/1     Running   0          108m   15.0.0.18   aks-agentpool-41427971-vmss000000   <none>           <none>
pod/green-deployment-5dbccd4bb9-59rk9   1/1     Running   0          108m   15.0.0.92   aks-agentpool-41427971-vmss000002   <none>           <none>
pod/green-deployment-5dbccd4bb9-kk9rq   1/1     Running   0          108m   15.0.0.46   aks-agentpool-41427971-vmss000001   <none>           <none>
pod/red-deployment-74b4d99d5c-2m6m9     1/1     Running   0          111m   15.0.0.23   aks-agentpool-41427971-vmss000000   <none>           <none>
pod/red-deployment-74b4d99d5c-5btzj     1/1     Running   0          111m   15.0.0.79   aks-agentpool-41427971-vmss000002   <none>           <none>
pod/red-deployment-74b4d99d5c-bxt4n     1/1     Running   0          111m   15.0.0.50   aks-agentpool-41427971-vmss000001   <none>           <none>
pod/white-deployment-69f8898c-28gjn     1/1     Running   0          104m   15.0.0.90   aks-agentpool-41427971-vmss000002   <none>           <none>
pod/white-deployment-69f8898c-84ssm     1/1     Running   0          104m   15.0.0.15   aks-agentpool-41427971-vmss000000   <none>           <none>
pod/white-deployment-69f8898c-shmhs     1/1     Running   0          104m   15.0.0.61   aks-agentpool-41427971-vmss000001   <none>           <none>

NAME                    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE    SELECTOR

service/blue-service    ClusterIP   10.0.9.185     <none>        8080/TCP   106m   app=blue
service/green-service   ClusterIP   10.0.201.73    <none>        8080/TCP   108m   app=green
service/red-service     ClusterIP   10.0.240.207   <none>        8080/TCP   111m   app=red
service/white-service   ClusterIP   10.0.12.67     <none>        8080/TCP   104m   app=white

NAME                                             HOSTS                                                                              ADDRESS         PORTS   AGE

ingress.extensions/colors-fanout-ingress         colors.penguintrails.com                                                           52.149.191.58   80      97m
ingress.extensions/colors-virtual-host-ingress   red.penguintrails.com,green.penguintrails.com,blue.penguintrails.com + 1 more...   52.149.191.58   80      106s


Connectivity Validations:

I've created a busybox container to run all the tests.  Let's validate connectivity to red pods and cluster serivce.

kubectl exec -it pingtest-b4b6f8cf-cxzt2 sh
/ # ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
40: eth0@if41: <BROADCAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue qlen 1000
    link/ether 9e:63:52:64:73:fc brd ff:ff:ff:ff:ff:ff
    inet 15.0.0.19/16 scope global eth0

       valid_lft forever preferred_lft forever


/ # wget -qO- 15.0.0.23:8080

red
/ # wget -qO- 15.0.0.79:8080
red
/ # wget -qO- 15.0.0.50:8080
red
/ # wget -qO- 10.0.240.207:8080
red
/ # exit

App Gateway Side

Fanout: 

Listeners-Fanout



Path-Based-Rules-Fanout



Backend-pool-fanout


Red-Pool

Front-end-IP


HTTP-Setting




Virtual Host:


















No comments: