Architecture Diagram
 |
| Azure Application Gateway ingress Controller in AKS |
Overview
This blog is inspired by one of my recent partner
engagements. There’s a fantastic article here about
this new Azure solution announcement. Let's understand the need for ingress controllers first. In my previous post, we took a closer look at Kubenet networking and services in AKS. When you create a Service of the type LoadBalancer, an underlying load balancer resource is created in Azure. The load balancer is configured to distribute traffic to the pods on a given port. This Load Balancer works at layer 4 and the service is unaware of the higher layer applications (http/https). Ingress controller works at layer 7 and can use more intelligent rules to distribute application traffic and also conserver public IP space. The common use case is to route traffic to different applications based on the inbound URL.
In this post we drill a little deeper into the architecture, take a closer look at two types of Ingress services – Simple Fanout and Name Based Virtual hosting and take it for a test drive. Application gateway as ingress controller has direct access to all kubernetes pods and performs the layer 7 functions. In the previous blog I covered the in-cluster ingress controller with nginx and AKS.
Test Drive
Follow the link to test drive application gateway ingress controller in Azure Kubernetes service. Create your own sample deployments using the sample yaml manifests.Fanout
http://colors.penguintrails.comhttp://colors.penguintrails.com/red
http://colors.penguintrails.com/green
http://colors.penguintrails.com/blue
http://colors.penguintrails.com/white
Virtual Host
http://red.penguintrails.comhttp://green.penguintrails.com
http://blue.penguintrails.com
default-white
Github
Pre-requisites:
- A Working AKS cluster with Application Gateway. Detailed documentation here.
- Link to my github repo here. Git clone my repository and off you go..
#
# Create Namespace
kubectl create namespace colors-ns
#
# Deployments
#
kubectl apply -f red-deployment.yaml
kubectl apply -f green-deployment.yaml
kubectl apply -f blue-deployment.yaml
kubectl apply -f white-deployment.yaml
#
# Ingress
#
kubectl apply -f colors-fanout.yaml
kubectl apply -f colors-virtual-host.yaml
Pre-requisites
From Azure Documentation here
- AKS with Advanced Networking enabled
- Application Gateway v2 in the same virtual network as AKS
Ingress Concepts
Some ingress concepts here
A fanout configuration routes traffic from a single IP address to more than one Service, based on the HTTP URI being requested. An Ingress allows you to keep the number of load balancers down to a minimum. For example, a setup like:
foo.bar.com -> 178.91.123.132 -> / foo service1:4200
/ bar service2:8080
Name-based virtual hosts support routing HTTP traffic to multiple host names at the same IP address.
foo.bar.com --| |-> foo.bar.com service1:80
| 178.91.123.132 |
bar.foo.com --| |-> bar.foo.com service2:80
Simple fanout
A fanout configuration routes traffic from a single IP address to more than one Service, based on the HTTP URI being requested. An Ingress allows you to keep the number of load balancers down to a minimum. For example, a setup like:
foo.bar.com -> 178.91.123.132 -> / foo service1:4200
/ bar service2:8080
Name based virtual hosting
Name-based virtual hosts support routing HTTP traffic to multiple host names at the same IP address.
foo.bar.com --| |-> foo.bar.com service1:80
| 178.91.123.132 |
bar.foo.com --| |-> bar.foo.com service2:80
Validations
Lets follow the red-service all the way.
Fanout
kubectl describe ingress colors-fanout-ingress -n colors-ns
Name: colors-fanout-ingress
Namespace: colors-ns
Address: 52.149.191.58
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
colors.penguintrails.com
/red red-service:8080 (15.0.0.23:8080,15.0.0.50:8080,15.0.0.79:8080)
/green green-service:8080 (15.0.0.18:8080,15.0.0.46:8080,15.0.0.92:8080)
/blue blue-service:8080 (15.0.0.5:8080,15.0.0.65:8080,15.0.0.78:8080)
white-service:8080 (15.0.0.15:8080,15.0.0.61:8080,15.0.0.90:8080)
Annotations:
<snip>
Virtual Host
kubectl describe ingress colors-virtual-host-ingress -n colors-nsName: colors-virtual-host-ingress
Namespace: colors-ns
Address: 52.149.191.58
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
red.penguintrails.com
red-service:8080 (15.0.0.23:8080,15.0.0.50:8080,15.0.0.79:8080)
green.penguintrails.com
green-service:8080 (15.0.0.18:8080,15.0.0.46:8080,15.0.0.92:8080)
blue.penguintrails.com
blue-service:8080 (15.0.0.5:8080,15.0.0.65:8080,15.0.0.78:8080)
*
white-service:8080 (15.0.0.15:8080,15.0.0.61:8080,15.0.0.90:8080)
Annotations:
<snip>
AKS Nodes
kubectl get nodes -o wideNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
aks-agentpool-41427971-vmss000000 Ready agent 9d v1.15.10 15.0.0.4 <none> Ubuntu 16.04.6 LTS 4.15.0-1071-azure docker://3.0.10+azure
aks-agentpool-41427971-vmss000001 Ready agent 9d v1.15.10 15.0.0.35 <none> Ubuntu 16.04.6 LTS 4.15.0-1071-azure docker://3.0.10+azure
aks-agentpool-41427971-vmss000002 Ready agent 9d v1.15.10 15.0.0.66 <none> Ubuntu 16.04.6 LTS 4.15.0-1071-azure docker://3.0.10+azure
AKS Pods and Service
kubectl get pods,service,ingress -o wide -n colors-ns
kubectl get pods,service,ingress -o wide -n colors-ns
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/blue-deployment-69d4cf87cd-6bz5p 1/1 Running 0 106m 15.0.0.65 aks-agentpool-41427971-vmss000001 <none> <none>
pod/blue-deployment-69d4cf87cd-sx5l9 1/1 Running 0 106m 15.0.0.78 aks-agentpool-41427971-vmss000002 <none> <none>
pod/blue-deployment-69d4cf87cd-vksvv 1/1 Running 0 106m 15.0.0.5 aks-agentpool-41427971-vmss000000 <none> <none>
pod/green-deployment-5dbccd4bb9-25fk6 1/1 Running 0 108m 15.0.0.18 aks-agentpool-41427971-vmss000000 <none> <none>
pod/green-deployment-5dbccd4bb9-59rk9 1/1 Running 0 108m 15.0.0.92 aks-agentpool-41427971-vmss000002 <none> <none>
pod/green-deployment-5dbccd4bb9-kk9rq 1/1 Running 0 108m 15.0.0.46 aks-agentpool-41427971-vmss000001 <none> <none>
pod/red-deployment-74b4d99d5c-2m6m9 1/1 Running 0 111m 15.0.0.23 aks-agentpool-41427971-vmss000000 <none> <none>
pod/red-deployment-74b4d99d5c-5btzj 1/1 Running 0 111m 15.0.0.79 aks-agentpool-41427971-vmss000002 <none> <none>
pod/red-deployment-74b4d99d5c-bxt4n 1/1 Running 0 111m 15.0.0.50 aks-agentpool-41427971-vmss000001 <none> <none>
pod/white-deployment-69f8898c-28gjn 1/1 Running 0 104m 15.0.0.90 aks-agentpool-41427971-vmss000002 <none> <none>
pod/white-deployment-69f8898c-84ssm 1/1 Running 0 104m 15.0.0.15 aks-agentpool-41427971-vmss000000 <none> <none>
pod/white-deployment-69f8898c-shmhs 1/1 Running 0 104m 15.0.0.61 aks-agentpool-41427971-vmss000001 <none> <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/blue-service ClusterIP 10.0.9.185 <none> 8080/TCP 106m app=blue
service/green-service ClusterIP 10.0.201.73 <none> 8080/TCP 108m app=green
service/red-service ClusterIP 10.0.240.207 <none> 8080/TCP 111m app=red
service/white-service ClusterIP 10.0.12.67 <none> 8080/TCP 104m app=white
NAME HOSTS ADDRESS PORTS AGE
ingress.extensions/colors-fanout-ingress colors.penguintrails.com 52.149.191.58 80 97m
ingress.extensions/colors-virtual-host-ingress red.penguintrails.com,green.penguintrails.com,blue.penguintrails.com + 1 more... 52.149.191.58 80 106s
Connectivity Validations:
I've created a busybox container to run all the tests. Let's validate connectivity to red pods and cluster serivce.
/ # ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
40: eth0@if41: <BROADCAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue qlen 1000
link/ether 9e:63:52:64:73:fc brd ff:ff:ff:ff:ff:ff
inet 15.0.0.19/16 scope global eth0
valid_lft forever preferred_lft forever
/ # wget -qO- 15.0.0.23:8080
red
/ # wget -qO- 15.0.0.79:8080
red
/ # wget -qO- 15.0.0.50:8080
red
/ # wget -qO- 10.0.240.207:8080
red
/ # exit
App Gateway Side
Fanout:
 |
| Listeners-Fanout |
 |
| Path-Based-Rules-Fanout |
 |
| Backend-pool-fanout |
 |
| Red-Pool |
 |
| Front-end-IP |
 |
| HTTP-Setting |
Virtual Host:
