In this post I discuss some of the changes you might need to make in integration test code that uses WebApplicationFactory<> or TestServer when upgrading to ASP.NET Core 3.0.
One of the biggest changes in ASP.NET Core 3.0 was converting it to run on top of the Generic Host infrastructure, instead of the WebHost. I've addressed that change a couple of times in this series, as well is in my series on exploring ASP.NET Core 3.0. This change also impacts other peripheral infrastructure like the TestServer used for integration testing.
Integration testing with the Test Host and TestServer
ASP.NET Core includes a library Microsoft.AspNetCore.TestHost which contains an in-memory web host. This lets you send HTTP requests to your server without the latency or hassle of sending requests over the network.
The terminology is a little confusing here - the in-memory host and NuGet package is often referred to as the "TestHost" but the actual class you use in your code is
TestServer. The two are often used interchangeably.
In ASP.NET Core 2.x you could create a test server by passing a configured instance of IWebHostBuilder to the TestServer constructor:
public class TestHost2ExampleTests
{
[Fact]
public async Task ShouldReturnHelloWorld()
{
// Build your "app"
var webHostBuilder = new WebHostBuilder()
.Configure(app => app.Run(async ctx =>
await ctx.Response.WriteAsync("Hello World!")
));
// Configure the in-memory test server, and create an HttpClient for interacting with it
var server = new TestServer(webHostBuilder);
HttpClient client = server.CreateClient();
// Send requests just as if you were going over the network
var response = await client.GetAsync("/");
response.EnsureSuccessStatusCode();
var responseString = await response.Content.ReadAsStringAsync();
Assert.Equal("Hello World!", responseString);
}
}
In the example above, we create a basic WebHostBuilder that returns "Hello World!" to all requests. We then create an in-memory server using TestServer:
var server = new TestServer(webHostBuilder);
Finally, we create an HttpClient that allows us to send HTTP requests to the in-memory server. You can use this HttpClient exactly as you would if you were sending requests to an external API:
var client = server.CreateClient();
var response = await client.GetAsync("/");
In .NET core 3.0, this pattern is still the same generally, but is made slightly more complicated by the move to the generic host.
TestServer in .NET Core 3.0
To convert your .NET Core 2.x test project to .NET Core 3.0, open the test project's .csproj, and change the <TargetFramework> element to netcoreapp3.0. Next, replace the <PackageReference> for Microsoft.AspNetCore.App with a <FrameworkReference>, and update any other package versions to 3.0.0.
If you take the exact code written above, and convert your project to a .NET Core 3.0 project, you'll find it runs without any errors, and the test above will pass. However that code is using the old WebHost rather than the new generic Host-based server. Lets convert the above code to use the generic host instead.
First, instead of creating a WebHostBuilder instance, create a HostBuilder instance:
var hostBuilder = new HostBuilder();
The HostBuilder doesn't have a Configure() method for configuring the middleware pipeline. Instead, you need to call ConfigureWebHost(), and call Configure() on the inner IWebHostBuilder. The equivalent becomes:
var hostBuilder = new HostBuilder()
.ConfigureWebHost(webHost =>
webHost.Configure(app => app.Run(async ctx =>
await ctx.Response.WriteAsync("Hello World!")
)));
After making that change, you have another problem - the TestServer constructor no longer compiles:
The TestServer constructor takes an IWebHostBuilder instance, but we're using the generic host, so we have an IHostBuilder. It took me a little while to discover the solution to this one, but the answer is to not create a TestServer manually like this at all. Instead you have to:
- Call
UseTestServer()insideConfigureWebHostto add theTestServerimplementation. - Build and start an
IHostinstance by callingStartAsync()on theIHostBuilder - Call
GetTestClient()on the startedIHostto get anHttpClient
That's quite a few additions, so the final converted code is shown below:
public class TestHost3ExampleTests
{
[Fact]
public async Task ShouldReturnHelloWorld()
{
var hostBuilder = new HostBuilder()
.ConfigureWebHost(webHost =>
{
// Add TestServer
webHost.UseTestServer();
webHost.Configure(app => app.Run(async ctx =>
await ctx.Response.WriteAsync("Hello World!")));
});
// Build and start the IHost
var host = await hostBuilder.StartAsync();
// Create an HttpClient to send requests to the TestServer
var client = host.GetTestClient();
var response = await client.GetAsync("/");
response.EnsureSuccessStatusCode();
var responseString = await response.Content.ReadAsStringAsync();
Assert.Equal("Hello World!", responseString);
}
}
If you forget the call to
UseTestServer()you'll see an error like the following at runtime:System.InvalidOperationException : Unable to resolve service for type 'Microsoft.AspNetCore.Hosting.Server.IServer' while attempting to activate 'Microsoft.AspNetCore.Hosting.GenericWebHostService'.
Everything else about interacting with the TestServer is the same at this point, so you shouldn't have any other issues.
Integration testing with WebApplicationFactory
Using the TestServer directly like this is very handy for testing "infrastructural" components like middleware, but it's less convenient for integration testing of actual apps. For those situations, the Microsoft.AspNetCore.Mvc.Testing package takes care of some tricky details like setting the ContentRoot path, copying the .deps file to the test project's bin folder, and streamlining TestServer creation with the WebApplicationFactory<> class.
The documentation for using WebApplicationFactory<> is generally very good, and appears to still be valid for .NET Core 3.0. However my uses of WebApplicationFactory were such that I needed to make a few tweaks when I upgraded from ASP.NET Core 2.x to 3.0.
Adding XUnit logging with WebApplicationFactory in ASP.NET Core 2.x
For the examples in the rest of this post, I'm going to assume you have the following setup:
- A .NET Core Razor Pages app created using
dotnet new webapp - An integration test project that references the Razor Pages app.
You can find an example of this in the GitHub repo for this post.
If you're not doing anything fancy, you can use the WebApplicationFactory<> class in your tests directly as described in the documentation. Personally I find I virtually always want to customise the WebApplicationFactory<>, either to replace services with test versions, to automatically run database migrations, or to customise the IHostBuilder further.
One example of this is hooking up the xUnit ITestOutputHelper to the fixture's ILogger infrastructure, so that you can see the TestServer's logs inside the test output when an error occurs. Martin Costello has a handy NuGet package, MartinCostello.Logging.XUnit that makes doing this a couple of lines of code.
The following example is for an ASP.NET Core 2.x app:
public class ExampleAppTestFixture : WebApplicationFactory<Program>
{
// Must be set in each test
public ITestOutputHelper Output { get; set; }
protected override IWebHostBuilder CreateWebHostBuilder()
{
var builder = base.CreateWebHostBuilder();
builder.ConfigureLogging(logging =>
{
logging.ClearProviders(); // Remove other loggers
logging.AddXUnit(Output); // Use the ITestOutputHelper instance
});
return builder;
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
// Don't run IHostedServices when running as a test
builder.ConfigureTestServices((services) =>
{
services.RemoveAll(typeof(IHostedService));
});
}
}
This ExampleAppTestFixture does two things:
- It removes any configured
IHostedServices from the container so they don't run during integration tests. That's often a behaviour I want, where background services are doing things like pinging a monitoring endpoint, or listening/dispatching messages to RabbitMQ/KafKa etc - Hook up the xUnit log provider using an
ITestOutputHelperproperty.
To use the ExampleAppTestFixture in a test, you must implement the IClassFixture<T> interface on your test class, inject the ExampleAppTestFixture as a constructor argument, and hook up the Output property.
public class HttpTests: IClassFixture<ExampleAppTestFixture>, IDisposable
{
readonly ExampleAppTestFixture _fixture;
readonly HttpClient _client;
public HttpTests(ExampleAppTestFixture fixture, ITestOutputHelper output)
{
_fixture = fixture;
fixture.Output = output;
_client = fixture.CreateClient();
}
public void Dispose() => _fixture.Output = null;
[Fact]
public async Task CanCallApi()
{
var result = await _client.GetAsync("/");
result.EnsureSuccessStatusCode();
var content = await result.Content.ReadAsStringAsync();
Assert.Contains("Welcome", content);
}
}
This test requests the home page for the RazorPages app, and looks for the string "Welcome" in the body (it's in an <h1> tag). The logs generated by the app are all piped to xUnit's output, which makes it easy to understand what's happened when an integration test fails:
[2019-10-29 18:33:23Z] info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
[2019-10-29 18:33:23Z] info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Development
...
[2019-10-29 18:33:23Z] info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint '/Index'
[2019-10-29 18:33:23Z] info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 182.4109ms 200 text/html; charset=utf-8
Using WebApplicationFactory in ASP.NET Core 3.0
On the face of it, it seems like you don't need to make any changes after converting your integration test project to target .NET Core 3.0. However, you may notice something strange - the CreateWebHostBuilder() method in the custom ExampleAppTestFixture is never called!
The reason for this is that WebApplicationFactory supports both the legacy WebHost and the generic Host. If the app you're testing uses a WebHostBuilder in Program.cs, then the factory calls CreateWebHostBuilder() and runs the overridden method. However if the app you're testing uses the generic HostBuilder, then the factory calls a different method, CreateHostBuilder().
To update the factory, rename CreateWebHostBuilder to CreateHostBuilder, change the return type from IWebHostBuilder to IHostBuilder, and change the base method call to use the generic host method. Everything else stays the same:
public class ExampleAppTestFixture : WebApplicationFactory<Program>
{
public ITestOutputHelper Output { get; set; }
// Uses the generic host
protected override IHostBuilder CreatHostBuilder()
{
var builder = base.CreateHostBuilder();
builder.ConfigureLogging(logging =>
{
logging.ClearProviders(); // Remove other loggers
logging.AddXUnit(Output); // Use the ITestOutputHelper instance
});
return builder;
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureTestServices((services) =>
{
services.RemoveAll(typeof(IHostedService));
});
}
}
Notice that the
ConfigureWebHostmethod doesn't change - that is invoked in both cases, and still takes anIWebHostBuilderargument.
After updating your fixture you should find your logging is restored, and your integration tests should run as they did before the migration to the generic host.
Summary
In this post I described some of the changes required to your integration tests after moving an application from ASP.NET Core 2.1 to ASP.NET Core 3.0. These changes are only required if you actually migrate to using the generic Host instead of the WebHost. If you are moving to the generic host then you will need to update any code that uses either the TestServer or WebApplicationFactory.
To fix your TestServer code, call UseTestServer() inside the HostBuilder.ConfigureWebHost() method. Then build your Host, and call StartAsync() to start the host. Finally, call IHost.GetTestClient() to retrieve an HttpClient that can call your app.
To fix your custom WebApplicationFactory, make sure you override the correct builder method. If your app uses the WebHost, override the CreateWebHostBuilder method. After moving to the generic Host, override the CreateWebHostBuilder method.