Improve gRPC service availability and efficiency with Traffic Director

Traffic Director is Google Cloud’s fully managed traffic control plane for service mesh and helps you manage traffic flows between gRPC-based services. For example, you can split traffic between multiple versions of your services for canary or blue/green deployments. You can manage this centrally (for example, by issuing API calls to Traffic Director) without having to re-deploy your applications.

With the most recent release of gRPC, we are bringing new traffic management capabilities to proxyless gRPC services that use Traffic Director: retries and session affinity.

Retry

Retries help you improve your service availability by enabling your gRPC applications to retry outbound requests according to a policy. This can be helpful if instances or endpoints (that host your server application(s)) are slow or flaky.

When your client gRPC application sends a request, the request gets resolved to a particular backend service based on request characteristics (for example, the hostname). You can use Traffic Director to set a retry policy for that backend service. The client will check the policy before sending a request to that backend service. The policy defines how your proxyless gRPC client application(s) can retry requests:

according to retry conditions (for example, when encountering response codes such as `unavailable`, `resource-exhausted`, and/or others)

x number of times where x is configurable (for example, retry up to 3 times)

Session Affinity

Session affinity is critical to enabling particular use cases and can also help you improve application performance and efficiency. It works by attempting to send requests to the same backend service instance or endpoint based on the request characteristics (HTTP headers).

For example, session affinity allows you to route requests from a particular user to the same shopping cart instance during a session so that the cart can return the items that are in the user’s cart. Additionally, because the instance can store those items in memory, it doesn’t need to repeatedly read and write entries from a database (which can introduce latency).

You can use Traffic Director to enable header-based session affinity for a backend service. When your gRPC application sends a request, it will attempt to route all RPCs with a given request header to the same endpoint. The application just needs to use the same header on all RPCs in a given session.

If you’ve been thinking about adopting a service mesh, improving performance, or increasing the reliability of your system, proxyless gRPC and Traffic Director can help you do that. Once you have proxyless gRPC applications set up with Traffic Director, you can enable the above features on all your applications via API calls, as opposed to having to code each application independently. 

For more information about using gRPC with Traffic Director and these new features, see the following links:

Traffic Director with proxyless gRPC services overview

Advanced traffic management overview