Google Cloud offers a variety of load balancing solutions which simplify the management of networking infrastructure and support different types of backend services. Now available, the cross-region internal Application Load Balancer expands this coverage by providing integrations to load balance, geo-route, and automatically failover to backends in multiple regions. In this walkthrough we’ll focus on workloads in Google Kubernetes Engine (GKE). This enables the following capabilities:
Global load balancing: Support for backends in multiple regionsImproved performance, reliability and high availability: Distributing traffic across multiple regions, and automatic failover to services in other regionsGeo routing: Cloud DNS policy manager to route traffic to nearest healthy backendsManaged certificates support: Google-managed/Self-managed certificates
Initial project setup and enable required APIs
code_block<ListValue: [StructValue([(‘code’, ‘export PROJECT_ID=cross-region-lb-sandboxrngcloud config set project cross-region-lb-sandboxrngcloud services enable cloudresourcemanager.googleapis.com rntttcompute.googleapis.com rncontainer.googleapis.com’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433dff10>)])]>
VPC network setup
Components: VPC network with 2 subnets in us-central1 and us-east1, for two GKE clusters
code_block<ListValue: [StructValue([(‘code’, ‘gcloud compute networks create mc-net –subnet-mode=custom –mtu=1450 –bgp-routing-mode=globalrnrngcloud compute networks subnets create snet1 –range=192.168.0.0/20 –stack-type=IPV4_ONLY –network=mc-net –region=us-central1 –secondary-range=snet1-range1=6.0.0.0/20,snet1-range2=6.0.16.0/20 –enable-private-ip-google-accessrnrngcloud compute networks subnets create snet2 –range=192.168.16.0/20 –stack-type=IPV4_ONLY –network=mc-net –region=us-east1 –secondary-range=snet2-range1=6.0.32.0/20,snet2-range2=6.0.48.0/20 –enable-private-ip-google-accessrnrngcloud compute firewall-rules create iap-fw –direction=INGRESS –priority=1000 –network=mc-net –action=ALLOW –rules=tcp:22 –source-ranges=35.235.240.0/20’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433df460>)])]>
Create GKE clusters
code_block<ListValue: [StructValue([(‘code’, ‘gcloud iam service-accounts create gke-ap-sarnexport GSA_EMAIL=gke-ap-sa@${PROJECT_ID}.iam.gserviceaccount.comrnrngcloud projects add-iam-policy-binding ${PROJECT_ID} \rn –member serviceAccount:${GSA_EMAIL} \rn –role roles/container.adminrnrngcloud container clusters create-auto “mc1” –region “us-central1” –enable-private-nodes –enable-private-endpoint –master-ipv4-cidr “6.0.128.0/28” –enable-master-authorized-networks –enable-master-global-access –enable-master-global-accessrn–master-authorized-networks 192.168.0.0/16 –network “projects/${PROJECT_ID}/global/networks/mc-net” –subnetwork “projects/${PROJECT_ID}/regions/us-central1/subnetworks/snet1” –cluster-secondary-range-name “snet1-range1” –services-secondary-range-name “snet1-range2” –service-account gke-ap-sa@${PROJECT_ID}.iam.gserviceaccount.com –scopes=cloud-platform –asyncrnrngcloud container clusters create-auto “mc2” –region “us-east1” –enable-private-nodes –enable-private-endpoint –master-ipv4-cidr “6.0.128.16/28” –enable-master-authorized-networks –enable-master-global-access –master-authorized-networks 192.168.0.0/16 –network “projects/${PROJECT_ID}/global/networks/mc-net” –subnetwork “projects/${PROJECT_ID}/regions/us-east1/subnetworks/snet2” –cluster-secondary-range-name “snet2-range1” –services-secondary-range-name “snet2-range2″ –service-account gke-ap-sa@${PROJECT_ID}.iam.gserviceaccount.com –scopes=cloud-platform –async’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433df850>)])]>
Create a Bastion host, install tools and get cluster credentials
code_block<ListValue: [StructValue([(‘code’, ‘gcloud compute instances create mc-test –zone=us-central1-a –machine-type=e2-small –service-account=gke-ap-sa@${PROJECT_ID}.iam.gserviceaccount.com –scopes=https://www.googleapis.com/auth/cloud-platformrnrngcloud compute ssh –zone “us-central1-a” “mc-test” –tunnel-through-iaprnrnrnsudo apt-get install kubectl google-cloud-sdk-gke-gcloud-auth-pluginrnrngcloud container clusters get-credentials mc1 –region us-central1rngcloud container clusters get-credentials mc2 –region us-east1rnrnkubectl config rename-context gke_cross-region-lb-sandbox_us-central1_mc1 mc1rnkubectl config rename-context gke_cross-region-lb-sandbox_us-east1_mc2 mc2’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433dfb50>)])]>
Deploy sample GKE backends:
code_block<ListValue: [StructValue([(‘code’, ‘for i in 1 2; do kubectl –context mc${i} apply -f – <<EOFrnapiVersion: apps/v1rnkind: Deploymentrnmetadata:rn name: whereamirn labels:rn app: whereamirnspec:rn replicas: 3rn selector:rn matchLabels:rn app: whereamirn template:rn metadata:rn labels:rn app: whereamirn spec:rn topologySpreadConstraints:rn – maxSkew: 1rn topologyKey: topology.kubernetes.io/zonern whenUnsatisfiable: DoNotSchedulern containers:rn – name: frontendrn image: us-docker.pkg.dev/google-samples/containers/gke/whereami:v1.2.20rn ports:rn – containerPort: 8080rnEOFrndone’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433df0a0>)]), StructValue([(‘code’, ‘for i in 1 2; do kubectl –context mc${i} apply -f – <<EOFrnapiVersion: v1rnkind: Servicernmetadata:rn labels:rn app: whereamirn annotations:rn cloud.google.com/neg: ‘{“exposed_ports”: {“80”:{“name”: “whereami”}}}’rn name: whereamirnspec:rn ports:rn – port: 80rn protocol: TCPrn targetPort: 8080rn selector:rn app: whereamirnEOFrndone’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433dfbe0>)])]>
Configuring the Load Balancer
Create Proxy-only subnets
code_block<ListValue: [StructValue([(‘code’, ‘gcloud beta compute networks subnets create proxy-snet1 \rn –purpose=GLOBAL_MANAGED_PROXY \rn –role=ACTIVE \rn –region=us-central1 \rn –network=mc-net \rn –range=6.0.144.0/24rn rn gcloud beta compute networks subnets create proxy-snet2 \rn –purpose=GLOBAL_MANAGED_PROXY \rn –role=ACTIVE \rn –region=us-east1 \rn –network=mc-net \rn –range=6.0.145.0/24’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b433dfc10>)])]>
Create firewall rules to allow TCP application traffic and health checks to the backends
code_block<ListValue: [StructValue([(‘code’, ‘gcloud compute firewall-rules create fw-allow-health-check \rn –network=mc-net \rn –action=allow \rn –direction=ingress \rn –source-ranges=130.211.0.0/22,35.191.0.0/16 \rn –rules=tcp:8080rnrngcloud compute firewall-rules create fw-allow-proxy-only-subnet \rn –network=mc-net \rn –action=allow \rn –direction=ingress \rn –source-ranges=6.0.144.0/24,6.0.145.0/24 \rn –rules=tcp:8080’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b42e975e0>)])]>
Create the resources to set up the Load Balancer, components shown in the architecture diagram above
code_block<ListValue: [StructValue([(‘code’, ‘gcloud compute health-checks create http gil7-basic-check \rn –use-serving-port \rn –globalrnrngcloud compute backend-services create whereami \rn –load-balancing-scheme=INTERNAL_MANAGED \rn –protocol=HTTP \rn –enable-logging \rn –logging-sample-rate=1.0 \rn –health-checks=gil7-basic-check \rn –global-health-checks \rn –globalrnrnfor zone in $(gcloud container clusters describe mc1 –region us-central1 –format json | jq -r .locations[]); do gcloud compute backend-services add-backend whereami –global –balancing-mode=RATE –max-rate-per-endpoint=1000 –network-endpoint-group=whereami –network-endpoint-group-zone=${zone}; donernrnfor zone in $(gcloud container clusters describe mc2 –region us-east1 –format json | jq -r .locations[]); do gcloud compute backend-services add-backend whereami –global –balancing-mode=RATE –max-rate-per-endpoint=1000 –network-endpoint-group=whereami –network-endpoint-group-zone=${zone}; donernrngcloud compute url-maps create gil7-lb \rn –default-service=whereami \rn –globalrnrngcloud compute target-http-proxies create gil7-http-proxy \rn –url-map=gil7-lb \rn –globalrnrngcloud compute addresses create ip-central –region=us-central1 –subnet=snet1 –purpose=GCE_ENDPOINTrnrngcloud compute addresses create ip-east –region=us-east1 –subnet=snet2 –purpose=GCE_ENDPOINTrnrnexport IP_CENTRAL=$(gcloud compute addresses describe ip-central –region us-central1 –format json | jq -r .address)rnrnexport IP_EAST=$(gcloud compute addresses describe ip-east –region us-east1 –format json | jq -r .address)rnrngcloud compute forwarding-rules create gil7-fw-rule-central –load-balancing-scheme=INTERNAL_MANAGED –network=mc-net –subnet=snet1 –subnet-region=us-central1 –address=${IP_CENTRAL} –ports=80 –target-http-proxy=gil7-http-proxy –globalrnrngcloud compute forwarding-rules create gil7-fw-rule-east –load-balancing-scheme=INTERNAL_MANAGED –network=mc-net –subnet=snet2 –subnet-region=us-east1 –address=${IP_EAST} –ports=80 –target-http-proxy=gil7-http-proxy –global’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b42e974c0>)])]>
Create DNS zone with geo-routing
code_block<ListValue: [StructValue([(‘code’, ‘gcloud dns managed-zones create cross-region-lb-internal –description=”” –dns-name=”cross-region-lb.apps.internal.” –visibility=”private” –networks=”https://www.googleapis.com/compute/v1/projects/${PROJECT_ID}/global/networks/mc-net”rnrngcloud dns record-sets create whereami.cross-region-lb.apps.internal –ttl=”30″ –type=”A” –zone=”cross-region-lb-internal” –routing-policy-type=”GEO” –routing-policy-data=”us-central1=gil7-fw-rule-central@global;us-east1=gil7-fw-rule-east@global” –enable-health-checking’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b417c49a0>)])]>
Test the Load Balancer and geo-routing
code_block<ListValue: [StructValue([(‘code’, ‘gcloud compute instances create mc-test –zone=us-east1-d –machine-type=e2-small’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b3eb723d0>)])]>
Querying the load balancer from a VM in each region shows that Cloud DNS routes the requests to the closest backends.
code_block<ListValue: [StructValue([(‘code’, ‘for zone in us-central1-a us-east1-d; do gcloud compute ssh mc-test –zone ${zone} — curl whereami.cross-region-lb.apps.internal; done’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b3eb72f70>)])]>
Test automatic failover
To test, scale down the deployment in one region. Running the same command as above, we can see that the requests failover to us-central1 now, since there are no healthy endpoints in us-east1 anymore.
code_block<ListValue: [StructValue([(‘code’, ‘kubectl scale –replicas 0 deploy/whereami –context mc2’), (‘language’, ”), (‘caption’, <wagtail.rich_text.RichText object at 0x3e2b3eb72e50>)])]>
Learn more
To learn more on this topic please checkout the links below.
Documentation: Set up a cross-region internal Application Load Balancer with hybrid connectivityDocumentation: Internal Application Load Balancer overviewDocumentation: DNS policies overview | Google Cloud
Cloud BlogRead More
