More Resilient Service-to-Service Communication with Polly and .NET HttpClientFactory

More Resilient Service-to-Service Communication with Polly and .NET HttpClientFactory

Using Polly, the resilience framework for .NET, you can gracefully handle the lost packets, thrown exceptions, and failed requests which inevitably make their way into service-to-service communications on the web. A previous post on the Twilio blog introduced Polly, the .NET resilience framework. This post builds on that knowledge to strengthen your application's interaction with API endpoints.

The original .NET HttpClient was a convenient way to send and receive HTTP requests and responses on the web, but unless HttpClient was carefully implemented its use could result in socket exhaustion. The release of .NET Core 2.1 introduced HttpClientFactory, which can be instantiated once and used throughout the life of an application.

This post will show you how to combine HttpClientFactory with Polly to achieve efficient and resilient service-to-service communications. You'll see how you can do it more reliably, conveniently, and with less code than using HttpClient.

Prerequisites

To follow along with this post you need:

.NET Core 2.1+ SDK (The SDK includes the runtime.)

Git (Used to clone the repository for this post.)

curl, Fiddler, Postman, or Insomnia (The Git installer for Windows includes a curl executable.)

Visual Studio 2017/2019, Visual Studio Code, or your favorite development environment.

The companion repository for this post is available on GitHub. You'll use it as the basis for the coding you'll do in the tutorial that follows.

Getting started

The companion repository contains two Visual Studio ASP.NET Core 2.1 Web API solutions, one implementing a Weather Service and the other a Temperature Service. The Weather Service has three action methods, one to get the weather information for a location, one to update weather information for a location and one to delete weather information for a location. The Weather Service in turn calls a highly unreliable Temperature Service. You can imagine that the Weather Service would also call other backend services like a humidity service, a storm tracker service, etc., but only the Temperature Service is shown in this example.  

You'll run the TemperatureService project to test the code, but all the changes you'll need to make to implement HttpClientFactory with Polly will be made to the WeatherService project.

Check out the code in TemperatureController.cs to see how the unreliable Temperature Service is implemented. In TemperatureController.cs all the methods return failures 75% of the time.

Under the PollyHttpClientFactoryWeatherService project directory, open the following solution files in separate instances of Visual Studio:

TemperatureService/TemperatureService.sln
WeatherService/WeatherService.sln

Using separate instances of Visual Studio (or separate console windows if you're running from the .NET CLI) will enable you to conveniently run both services simultaneously and watch the console output from each application.

In the WeatherService project, open the WeatherController.cs file. It has three methods that correspond to GET, POST, and DELETE HTTP methods. They call corresponding methods in the TemperatureController.cs file in the TemperatureServiceproject.

Note how the HttpClient is passed into the controller class through dependency injection in the constructor. HttpClient will change to HttpClientFactory in later modifications to the code.

Test the broken services

Try out the existing, unmodified code to see the Weather Service returning HTTP errors based on the errors it receives from the Temperature Service. The unmodified code doesn't include Polly, so it doesn't have the resilience features that would handle these failures automatically.

Start the two services from their separate instances of Visual Studio (or from the .NET CLI if you're working with another IDE or an editor). The Weather Service opens on HTTP port 5001 and the Temperature Service opens on port 6001.

The following instructions for making the HTTP calls to the Weather Service use curl, but you can certainly use Fiddler, Postman, Insomnia, or PowerShell Invoke-WebRequest if you prefer.

Open a console window. If you're a Windows user, you should use a Windows Command Prompt window rather than a PowerShell window unless you're familiar with the configuration of curl and PowerShell on your system.  Depending on your environment, you may have to adjust the syntax slightly to get them to work for you.

Try the HTTP GET request first.

Execute the following command-line instruction in your console window:

Your request will first hit the Weather Service, which then makes a request to the Temperature Service. The call to the Temperature Service will result in an HTTP 500 Internal Server Error response being returned to the Weather Service.

Execute the command three more times. You will get two more 500 error responses and the fourth time you'll get an HTTP 200 OK response along with an integer for temperature returned from the Temperature Service. If you're using a graphical tool to make the calls you'll see the temperature value in the Content field.

Here’s an example of how to make the same request in Fiddler: open the composer tab, enter http://localhost:5001/weather/11 in the address bar, make sure GET is selected in the dropdown list, and hit Execute four times. Your session should look like the following:

Try the HTTP DELETE method. Execute the following command-line instruction four times:

The first three times you execute the command it will return a 500 Internal Server Error and the fourth time will return a 200 OK response.

Using the POST is a little more involved. You have to set the Content-Type header to application/json and provide a body for the request, as follows:

The first three calls to this will return 500 errors and the fourth will return a 201 Created response.

To execute the DELETE method use the following command:

Again, the first three instances will fail with 500 Internal Server Errors and the fourth will succeed with a 200 OK.

Leave the console window (or application) open. You'll be using the same commands to test the Polly and HttpClientFactory functionality later on.

Adding HttpClientFactory and Polly Policies

Enough with broken code, let's fix it all. (Well, mostly, but you'll come to that later.)

If you don’t want to code along, execute the following command in the PollyHttpClientFactoryWeatherService directory to checkout the finished code:

There will be changes to three files in the WeatherService project: WeatherService.csproj, Startup.cs, and WeatherController.cs.

If you're coding along with these instructions, here's an overview of what your going to do:

In the Startup class for the Weather Service project you are going to define the Polly policies, the HttpClientFactory that returns HTTP clients, and a policy selector to choose the right policy based on the request.

At the point you use Polly to make a request in the controller methods your code will go from this: 

To this:

You can't even tell Polly is there! Read on to find out how. 

In the WeatherService project, add the Microsoft.Extensions.Http.Polly NuGet package, being sure to pick the version that works with the version of .NET Core you are using. The source code provided in the companion repository uses .NET Core 2.1, so the appropriate version of the Polly NuGet package is  version 2.1.1.

In Startup.cs add a using Polly; statement at the top of the file.

Add three Polly policies to the ConfigureServices method by inserting the following C# code immediately before the end of the method:

The first policy retries failed requests immediately and up to three times.

The second policy retries failed requests up to three times, but adds a delay between each retry.

The final policy does nothing: it lets the request and response pass right through it without affecting them in any way. This one will be used for POST calls, because POST calls are not idempotent and you should not retry a failed request in all scenarios.

Adding HttpClientFactory is very easy: in this case you'll be adding a named HttpClient (as opposed to a typed one), setting the base address, and adding a single header.

Add the following C# code immediately after the code you added in the previous step:

Along with injection an instance of the HttpClient class into a controller, HttpClientFactory allows us to select a Polly policy to apply to a given request.

The code below selects the polices defined above based on the HTTP verb, but you can use any criteria that suits, another popular one is the URL of the request.

Add the following C# code at the end of the line of the previous code block immediately following the closing parenthesis ( ) ) and overwriting the comment:

That’s almost everything fixed, except for a few minor changes to WeatherController.cs.

Remove the _httpClient private field and change the constructor to take an instance of HttpClientFactory as an argument by replacing the existing code with the following:

Inside the GET method, remove the line that references the _httpClient. Instead, use the HttpClientFactory to request a HttpClient and then make the request to the Temperature Service. Note how there is no reference to Polly or the policies; there isn't even a using statement!

To fix up the POST method add the following line to the top of the method:

Locate the call to the Temperature Service shown in the following line of code:

Replace the code above with the following:

For the DELETE method, add the following C# code to the beginning of the method:

Find the existing call to the Temperature Service shown below:

Replace the code above with the following C# code:

That’s it, you’re done. Try it out!

Testing Polly with HttpClientFactory

You can see the new code in action by repeating the same curl commands you used to see what happens with "brittle" service-to-service communication. This time your Weather Service is built to handle failed requests to the unreliable Temperature Service.

When you try HTTP GET and DELETE requests to the Weather Service you will get success responses because the HttpClientFactory is applying the Polly Retry and Wait & Retry policies to the requests. You'll see there is a noticeable delay in the GET responses because the Polly Wait & Retry policy configured in Startup.cs specifies an interval before each call is repeated. With three retries, that adds up to a perceptible delay. Wait & Retry is a useful strategy when calling services that experience transient spikes in utilization; waiting allows time for the service to be able to process requests.

If you try the POST you will see the same failure rate because the Polly NoOpAsync (No Operation) policy is being used. Polly doesn't take any action when such requests fail.

If you have built a new .NET Core 2.1 or 2.2 application without Polly—but with HttpClientFactory—you can see how easy it would be to retrofit. Make all your changes in the ConfigureServices method and your done.

Summary

With the arrival of the HttpClientFactory in .NET Core 2.1, using Polly has become even easier; resilience policies are applied to requests from the HttpClient through the dependency injection and the right policy can be chosen for a given request with a policy selector. All code is isolated in the Startup class and you even don’t need to reference Polly where you make the HTTP requests!

Additional Resources

Companion Repository – The complete code for the projects in this post is available on GitHub and can be reused under the MIT license.

Bryan’s blog posts on Polly - Check out some of Bryan’s posts on Polly on his own blog.

The Polly Project – The project homepage is an essential resource for new feature announcements and other Polly news. Check out the elevator pitch while you’re there.

The Polly repo on GitHub – The source, issues, and essential usage instructions.

Polly in the NuGet Gallery – All the installation goodness.

Bryan Hogan is a software architect, podcaster, blogger, Pluralisight author, and speaker who has been working in .NET since 2004. He lives in the Boston area and is involved with the community there through meetups and other events. Last year Bryan authored a Pluralsight course on Polly, he also blogs at the No Dogma Blog and even hosts a podcast!