Operational goodies for your IPv4/IPv6 dual-stack Kubernetes clusters

By mullaned2002

July 12, 2023

188

Are you ready to start your IPv6 journey? Is your cloud provider ready to start it with you? Google Kubernetes Engine (GKE) supports dual-stack Kubernetes clusters to help your journey to IPv6 while ensuring your applications are v6-ready. And to address the operational demands for IPv6 workloads, we’re adding several features to GKE networking to expand protection for both inbound and outbound IPv6 traffic, making them more highly available, secure, and observable.

The following features for dual-stack GKE clusters are now IPv6-aware, making it easier to enable v6 workloads with solutions that use both v6 and v4 Pods:

Load Balancer Services

FQDN Network Policies

Dataplane V2 observability

These new features complement the extensive work we’ve been doing for GKE to support IPv6 at the same level as we do IPv4. For example:

Dual-stack clusters – We’ve supported IPv4 and IPv6 front-ends with Ingress for some time, and our managed Gateway API has supported them since it launched. As of December 22nd, 2022, dual-stack GKE clusters have been available with global unicast addresses (GUA) as well as unique local addresses (ULA) on Google Cloud VPC networks. With GKE’s dual-stack clusters, both nodes and Pods get an IPv4 and IPv6 address to enable communication with both IP address families.

DNS support – GKE supports both IP address families with multiple DNS solutions. From inception, kube-dns supports dual-stack with both A and AAAA records. GKE also provides a more robust, scalable and performant DNS service through Cloud DNS. This Google Cloud-native DNS integration includes in-cluster name resolution with full support of IPv4 and IPv6 records.

Dual-stack Kubernetes Services – For Services, either single-stack IPv4, single-stack IPv6, or dual-stack addresses can be allocated. When we released dual-stack clusters, we supported clusterIP and nodePort Services. These fundamental constructs enable IPv6-capable Kubernetes workloads to be connected in a cluster.

Serving IPv6 to the world – GKE clusters have long been able to expose your workloads in a highly available manner through Kubernetes Ingress services on Google Cloud. By deploying your Gateway and Ingress services on GKE, you get the benefits of Google Networking at the edge to serve and protect with IPv6! Both Kubernetes Gateway API and Ingress on GKE use our tried-and-true Google Cloud Load Balancers, giving you the assurance of proven infrastructure. Additionally, while serving IPv6 to the world, you can protect your applications with Google Cloud Cloud Armor security policies. For example, you can reference your Cloud Armor security policy on your Gateway or the Backend Configuration CRD on your Ingress to define allow and deny lists with IPv6 addresses.

Now, let’s take a look at the latest IPv6 features and capabilities we’ve developed for GKE.

GKE Load Balancer Services

We’re excited to announce that the Service type LoadBalancer is now available with dual-stack capabilities. This means you will be able to create Kubernetes LoadBalancer Services and specify their IP families. As a benefit of running GKE, these are deployed as Google Cloud Network Load Balancers, which can be addressed either publicly or privately with the IP address family of your choice (i.e. IPv4-only, IPv6-only, or both).

Here’s an example of a YAML that you can use to create a dual-stack Kubernetes LoadBalancer Service on GKE exposed as a Google Cloud Network Load Balancer:

code_block[StructValue([(u’code’, u’apiVersion: v1rnkind: Servicernmetadata:rn name: server1-l4xlb-dualrn namespace: defaultrnspec:rn ipFamilyPolicy: RequireDualStackrn ipFamilies:rn – IPv4rn – IPv6rn ports:rn – port: 80rn targetPort: 80rn protocol: TCPrn selector:rn app: server1rn type: LoadBalancer’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e94830c0090>)])]

Once you’ve created a dual-stack Kubernetes LoadBalancer Service, you can confirm that both an IPv4 and IPv6 address have been assigned to the Service:

code_block[StructValue([(u’code’, u’$ kubectl get service server1-l4xlb-dualrnrnNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGErnserver1-l4xlb-dual LoadBalancer 10.180.9.96 35.232.168.42,2600:1900:4000:b1b8:8000:0:0:0 80:30754/TCP 4m56s’), (u’language’, u”), (u’caption’, <wagtail.wagtailcore.rich_text.RichText object at 0x3e94830c0510>)])]

You can use the standard Kubernetes API to create dual-stack Load Balancers and apply GKE annotations as you wish.

GKE FQDN Network Policies

We are advancing our capabilities for GKE with dual-stack support for fully qualified domain name (FQDN) Network Policies. This exciting feature advances the Network Security posture of workloads deployed on GKE to account for IPv6-capable applications.

By leveraging both A and AAAA records, FQDN Network Policies seamlessly provides advanced network security for both IPv4 and IPv6 address families. FQDN Network Policies enforce egress traffic policies when a workload reaches out to specific destinations that are outside of GKE cluster(s) resolving as IPv4 or IPv6 addresses. FQDN is additive to any existing endpoints allowed by the egress Network Policy. Once FQDN Network Policies are created and applied as an egress policy, an implicit DENY is applied for all endpoints that are not specified as an allowlisted destination.

These capabilities provide network security consistency across both IPv4 and IPv6 as you bring your IPv6-capable workloads onto GKE.

GKE Dataplane V2 observability

Opening up a world of metrics — our GKE Dataplane V2 observability launch brings visibility into your IP4/IPv6 workloads. This feature set includes metrics and troubleshooting tools to make your dual-stack GKE clusters operationally ready. The GKE Dataplane V2 observability stack enables you to have dual-stack Pod traffic metrics for the network info you care about. You can use Cloud Monitoring Metrics Explorer to monitor Dataplane V2 metrics for your IPv6 workloads, while our Managed Hubble solution for IPv6 Kubernetes workloads on GKE lets you troubleshoot the environment. The open source Hubble project is an observability platform built on Cilium and eBPF. Built for GKE’s Dataplane V2, our Managed Hubble UI gives you visibility into connection information and Network Policy enforcement in the form of a service map and a Network Policy verdict table. Finally, a CLI for interactive live troubleshooting lets you better understand your dual-stack Kubernetes workloads.

Get ready for dual-stack GKE Clusters

We hear from our users that dual-stack clusters are the stepping stones to an IPv6-only world. Together, this suite of features improves the operational readiness of your Kubernetes workloads for IPv6. Going to production with IPv6 implicitly means showing operational readiness in terms of high-availability, security, and observability. These releases should increase your confidence running dual-stack workloads on GKE.

For further reading, check out these resources on our current dual-stack capabilities.