Improving logging performance with source generators: Exploring .NET 6 - Part 8

In this post I look at the new LoggerMessage source generator, showing how and when to use it. I start by showing the existing ways to use ILogger, pitfalls to watch out for, and approaches to improve performance. I then show how the source generator simplifies this. We look at the code it generates, and do some simple benchmarks to see the performance benefits this brings.

Source generators

One of the big features that shipped in .NET 5 was source generators. These are extremely powerful features that enable you to hook into the compilation process, generating code at compile time for performance or ergonomic reasons.

Although source generators shipped with .NET 5, they weren't really exploited much by the core .NET libraries, or by ASP.NET Core. In .NET 6, that's all changed, with significant investments being made in that area. For example:

• There's a new source generator to help with logging using ILogger (the focus of this post)
• You can use source generators with System.Text.Json to reduce the runtime overhead of serializing to and from your entities.
• Blazor/Razor compilation is now a source generator, instead of being an explicit two step process, enabling features like Hot Reload (this actually causes me some problems unfortunately, as I described in a previous post)
• You can use the Event source generator to simplify creating an EventSource

Writing a source generator still isn't quite as simple as I wish it were, with many edge cases and caveats you can run into. Nevertheless, they have the real potential to significantly reduce boilerplate in your code, or to improve runtime performance by avoiding reflection and generating code at compile time instead.

In this post I focus on the LoggerMessage source generator. This generator has the potential to increase performance by reducing boilerplate. To understand how and why it does this, we'll start by looking at ILogger, some of the incorrect ways to use it, and the most performant ways to use it.

Logging a message with ILogger

If you've used ASP.NET Core before, you'll no doubt be familiar with ILogger. This is the "universal" logging abstraction built into ASP.NET Core that separates the creating of log messages from where they are written. This isn't a new concept, logging frameworks like log4Net, NLog, and Serilog have always taken this approach. The difference is that ILogger aims to be a "lowest common denominator" for logging libraries. That means your library dependencies can use ILogger, making it easier for them to write log messages that end up in your standard app logs.

Previously, without a "universal" abstraction, this was harder than you might think. The (now deprecated) LibLog library was a great solution to this, but it always felt kind of like a hack, and the "defacto standard" ILogger abstraction is the recommended approach for library authors.

If you're in a typical ASP.NET Core application, the ILogger infrastructure will already be configured. To write a log message, you first need access to an ILogger or ILogger<T> instance. This can be obtained from the DI container through constructor injection. For example, the following is a super-simple MVC app, built using .NET 6's minimal APIs:

using Microsoft.AspNetCore.Mvc;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
var app = builder.Build();

app.MapControllers();
app.Run();

Below is a controller for the default route (/) that returns "Hello World!". An ILogger<T> instance is injected into the constructor, and is used to write a log message when the API is called:

public class TestController
{
    private readonly ILogger<TestController> _logger;
    public TestController(ILogger<TestController> logger)
    {
        _logger = logger;
    }

    [HttpGet("/")]
    public string Get()
    {
        _logger.LogInformation("Writing hello world response");

        return "Hello world!";
    }
}

In the Console, you'll see the log message printed out (along with other startup log messages):

info: TestController[0]
      Writing hello world response

This is the simplest way to use ILogger, and for most cases, it's perfectly adequate. However ILogger has additional features, such as structural logging, which allows both for formatting of log messages, and, more importantly, efficient filtering and management of your logs when using something like Seq.

For example, imagine you had a simple Person record:

public record Person (int Id, string Name);

Which you want to include in the log message's output:

var person = new Person(123, "Joe Blogs");
_logger.LogInformation("Writing hello world response to {Person}", person);

This will write a log message like the following:

info: TestController[0]
      Writing hello world response to Person { Id = 123, Name = Joe Blogs }

These are both typical ways to use ILogger, now let's look at some of the problems with this, and ways to improve it!

1. Don't use string interpolation

String interpolation using $"" is generally great, and it makes for much more readable code than string.Format(), but you should almost never use it when writing log messages. So don't do this:

_logger.LogInformation($"Writing hello world response to {person}"); // Using interpolation instead of structured logging

The output of this method to the Console will be the same when using string interpolation or structural logging, but there are several problems with using string interpolation in your log messages:

1. You lose the "structured" logs, so when using log management tools like Seq, you won't be able to filter by the format values.
2. Similarly, you no longer have a constant "message template" to find all identical logs (unless you provide an explicit EventId, which basically noone does as far as I can tell!)
3. The serialization of person is done ahead of time, before the string is passed into LogInformation. If the log message is filtered out (maybe you're only logging Warning/Error level logs), then you've done all that work for nothing.

The first two points are good enough reasons not to use string interpolation in my opinion, but the third point is interesting as it highlights an issue with our "correct" usage: we're allocating the "message template" string every time we log the message, even if we're going to filter out the message later.

2. Check if the log level is enabled

Log level filtering is a standard feature of most logging frameworks. It allows you to control which log messages are kept, and which are ignored. For example, when developing locally, you might want to see Debug logs, so you can more easily track behaviour. However, in production, that would likely generate far too many log messages to store and be useful, so you typically filter out to only recording Information or even Warning level logs.

With that in mind, our typical logging usage has a problem:

_logger.LogInformation("Writing hello world response to {Person}", person);

Notice that we're allocating the message template string here, even though Information level logging might not be enabled! A better approach would be to check if the log level is enabled before trying to write the log message:

if (_logger.IsEnabled(LogLevel.Information))
{
    _logger.LogInformation("Writing hello world response to {Person}", person);
}

In this case, if the Information level logs aren't enabled, we won't even try to write the log message, saving us both time and allocations. The obvious downside with this approach is that it's extra boilerplate to include with every message. You also have to make sure that the log level you're checking (LogLevel.Information) matches the level you write (LogInformation), which is just more bugs to creep in.

This little bit of friction means that in most cases I've seen, people forgo the check, and just take the performance hit.

3. Make sure you pass the correct number of parameters

Let's say you decide to update the log message, to include a "reason" as to why the log is being written:

_logger.LogInformation("Writing hello world response to {Person} because {Reason}", person);

Do you see the mistake in the above code? I've added the {Reason} parameter to the log message, but I haven't passed a reason in the log message. This is a Very Bad Move™. At runtime, we'll now get an exception when this log message is called!

Luckily, tools like Rider can spot these sorts of errors and warn you about them:

Unfortunately other than that, if you're using the "normal" logging methods, there's not much else you can do.

4. Use LoggerHelper.Define<> to optimise template parsing

Structured logging is generally great, but there's no denying it's more work for the ILogger to do. Rather than just taking a string and passing it to some output, it must parse and tokenise the string, to identify where to insert the serialized objects. In the example we've been working with so far, that means extracting the {Person} from the message.

What's more, the ILogger must do this every time you call ILogger, even with the same message template. Now, there's obviously a certain amount of caching we can do (and which ILogger does do), but there's only so much caching you can do before it's a memory leak. In fact, ILogger currently caches the parsing results of up to 1024 message formats.

Rather than relying on the formatting being automatically cached, there's another option— do the formatting yourself. If you know that a log message is called often, and so you want to make sure the log formatting is as performant as possible, you can use the LoggerMessage.Define<> method.

I described this method in a previous post (4 years ago!) but to recap, this allows you to create a "helper" Action<> for your log message. For example, we could define a helper for our test method as:

private static readonly Action<ILogger, Person, Exception?> _logHelloWorld =
    LoggerMessage.Define<Person>(
        logLevel: LogLevel.Information,
        eventId: 0,
        formatString: "Writing hello world response to {Person}");

This defines an Action that takes three parameters, an ILogger instance, the Person to serialize, and an optional Exception object (the latter of which we will not use).

To use this method, we would update our callsite code to:

[HttpGet("/")]
public string Get()
{
    var person = new Person(123, "Joe Blogs");

    _logHelloWorld(_logger, person, null);

    return "Hello world!";
}

There are three main advantages to this

1. The template parsing is done once when in the LoggerMessage.Define() method, and cached for the lifetime of the app.
2. The _logHelloWorld action automatically calls _logger.IsEnabled() for us, so we don't have to use it at the call site.
3. We are forced to pass the correct type and number of arguments to the logger method (though this one is a bit questionable, as you could still create your message template incorrectly)

Using LoggerDefine pretty much gives us the best of all worlds. So why isn't it the "normal" way to write a log message? Well, for obvious reasons, it's a complete pain! There's so much ceremony and boilerplate required to create the Action<> that it's just too much friction for people to bother with most of the time.

And that's where the source generator comes in!

The .NET 6 [LoggerMessage] source generator

The fundamental role of the [LoggerMessage] source generator is to reduce the friction required for "best practice"/"high performance" logging. Specifically, it auto-generates the LoggerMessage.Define call for you!

In .NET 6, you can create a partial method, decorate it with the [LoggerMessage] attribute, and the source generator will automatically "fill in" the LoggerMessage.Define() call for you.

For example, going back to our TestController example, we can create the source generator version using:

public partial class TestController
{
    [LoggerMessage(0, LogLevel.Information, "Writing hello world response to {Person}")]
    partial void LogHelloWorld(Person person);
}

This uses a c# feature called partial methods, which allows you to define a method, but provide its implementation in another file. In this case, the implementation is generated by the source generator.

Note the we also had to mark the TestController as partial

To call the log method, you simply call the function like any other:

[HttpGet("/")]
public string Get()
{
    var person = new Person(123, "Joe Blogs");

    LogHelloWorld(person);

    return "Hello world!";
}

Behind the scenes, the [LoggerMessage] source generator generates the LoggerMessage.Define() code to optimise your method call. The following shows the generated code, tidied up a little by extracting namespaces etc to simplify things:

using System;
using Microsoft.Extensions.Logging;

partial class TestController 
{
    private static readonly Action<ILogger, Person, Exception?> __LogHelloWorldCallback 
        = LoggerMessage.Define<Person>(
            LogLevel.Information,
            new EventId(0, nameof(LogHelloWorld)), 
            "Writing hello world response to {Person}", 
            new LogDefineOptions() { SkipEnabledCheck = true });

    partial void LogHelloWorld(Person person)
    {
        if (_logger.IsEnabled(LogLevel.Information))
        {
            __LogHelloWorldCallback(_logger, person, null);
        }
    }
}

As you can see, this code is very similar to the LoggerMessage.Define() we created earlier! The main difference is that the source generator explicitly hoists the IsEnabled check, which I don't think is strictly necessary in this case, but doesn't hurt! The really nice thing: you didn't have to write any of this yourself!

Additional features in the source generator

The previous section shows what I think will be the most common usage of the source generator, but the generator comes with some extra features too.

Static instances

In the above section I showed one of the "default" ways of using the [LoggerMessage] source generator, in which it's defined as an instance method. When you do this, the source generator automatically "finds" the ILogger<T> field defined in the TestController, and uses it to call the LoggerMessage.Define() generated Action at runtime.

You don't have to use an instance method for your source generator, you can also create a static helper method and explicitly pass in the ILogger instance if you prefer:

[LoggerMessage(0, LogLevel.Information, "Writing hello world response to {Person}")]
static partial void LogHelloWorld(ILogger logger, Person person);

The resulting generated code is virtually identical; the only difference is the ILogger instance passed in the method is used instead of the field:

using System;
using Microsoft.Extensions.Logging;

partial class TestController 
{
    private static readonly Action<ILogger, Person, Exception?> __LogHelloWorldCallback 
        = LoggerMessage.Define<Person>(
            LogLevel.Information,
            new EventId(0, nameof(LogHelloWorld)), 
            "Writing hello world response to {Person}", 
            new LogDefineOptions() { SkipEnabledCheck = true });

    static partial void LogHelloWorld(ILogger logger, Person person)
    {
        // Note that the `logger` parameter instance is used instead
        // of the _logger field
        if (logger.IsEnabled(LogLevel.Information))
        {
            __LogHelloWorldCallback(logger, person, null);
        }
    }
}

The best use case for this is probably to create a specific static LogHelper class that contains all your log messages. Personally I think I prefer the instance approach, but I can see the appeal.

Message template checking

Another feature of the source generator, is that it's able to check for mismatched numbers of parameters at compile time, as the source generator is also an analyzer. This means that if you define a template with an incorrect number of arguments, e.g.

[LoggerMessage(0, LogLevel.Information, "Writing hello world response to {Person} with a {Reason}")]
partial void LogHelloWorld(Person person);

The source generator will spot that the message template includes two placeholders, but only one parameter is passed to the method, and will generate an error at compile time. This is a huge benefit, compared to throwing an exception at runtime!

Error SYSLIB1014 : Template 'Reason' is not provided as argument to the logging method
Error CS0103 : The name 'Reason' does not exist in the current context

This is a big advantage the source generator has over all the other approaches!

Dynamic log levels

Finally, the source generator is able to take a dynamic log level, if you need to change the level at runtime. Simply define your logging method with a LogLevel parameter, and the source generator will take this into account:

[LoggerMessage(Message = "Writing hello world response to {Person}")]
partial void LogHelloWorld(LogLevel logLevel, Person person);

You can then change the LogLevel each time you call the method:

var level = person.Name == "Dave" : LogLevel.Warning : LogLevel.Debug; 
LogHelloWorld(level, person);

One thing to be aware of though: dynamic log levels are not compatible with LoggerMessage.Define(). The source generator does its best to optimise this case anyway, but it's rather more complicated than using LoggerMessage.Define().

using System;
using System.Collections;
using System.Collections.Generic;
using Microsoft.Extensions.Logging;

partial class TestController 
{
    private readonly struct __LogHelloWorldStruct : IReadOnlyList<KeyValuePair<string, object?>>
    {
        private readonly Person _person;
        public __LogHelloWorldStruct(Person person)
        {
            this._person = person;

        }

        public override string ToString()
        {
            var Person = this._person;
            return $"Writing hello world response to {Person}";
        }

        public static string Format(__LogHelloWorldStruct state, global::System.Exception? ex) => state.ToString();

        public int Count => 2;

        public KeyValuePair<string, object?> this[int index]
        {
            get => index switch
            {
                0 => new KeyValuePair<string, object?>("Person", this._person),
                1 => new KeyValuePair<string, object?>("{OriginalFormat}", "Writing hello world response to {Person}"),

                _ => throw new global::System.IndexOutOfRangeException(nameof(index)),  // return the same exception LoggerMessage.Define returns in this case
            };
        }

        public IEnumerator<KeyValuePair<string, object?>> GetEnumerator()
        {
            for (int i = 0; i < 2; i++)
            {
                yield return this[i];
            }
        }

        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
    }

    partial void LogHelloWorld(LogLevel logLevel, Person person)
    {
        if (_logger.IsEnabled(logLevel))
        {
            _logger.Log(
                logLevel,
                new EventId(-1, nameof(LogHelloWorld)),
                new __LogHelloWorldStruct(person),
                null,
                __LogHelloWorldStruct.Format);
        }
    }
}

Of course, I've been making all these claims about performance, it's about time for me to put my money where my mouth is.

Benchmarking the different approaches.

I've been making lots of claims about performance, but I haven't been backing them up with any data, so I decided to do some simple benchmarking to prove my point. Using BenchmarkDotnet, I tested six scenarios:

• Using interpolated string log messages
• Using the default ILogger calls without wrapping the call in IsEnabled()
• Using the default ILogger calls with a guard IsEnabled() call
• Using LoggerMessage.Define() to create the log message
• Using the [LoggerMessage] source generator
• Using the [LoggerMessage] source generator with a dynamic log level

I then split each of the scenarios in two:

• Logging an Information message
• Logging a Debug message, where the minimum level is Information, so that the log message is dropped

The results are below, which I'll briefly discuss afterwards (You can see the code for the benchmarks on GitHub):

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.19042.1288 (20H2/October2020Update)
Intel Core i7-7500U CPU 2.70GHz (Kaby Lake), 1 CPU, 4 logical and 2 physical cores
.NET SDK=6.0.100-rc.2.21505.57
  [Host]     : .NET 6.0.0 (6.0.21.48005), X64 RyuJIT
  DefaultJob : .NET 6.0.0 (6.0.21.48005), X64 RyuJIT

• As expected, using string interpolation is terrible. It's much slower, and allocates loads, whether the log message actually gets written or not. Don't do it!
• Wrapping the log call in IsEnabled() is faster and removes all allocation when the log level is not enabled.
• Using LoggerMessage.Define() or the source generator are pretty much the same. They give the fastest results and are allocation free both when the log level is enabled or when it's disabled!
• When using a dynamic log level, you don't save on allocations, but the performance is just as good as the LoggerMessage.Define() case. If you need dynamic log levels, that's clearly a good trade off.

These results really highlight to me how powerful this source generator is. It dramatically reduces the friction associated with optimal logging practices, meaning there's very little reason not to use it. Hopefully teams will see it the same way!

Summary

In this post I described in detail the various ways to write log messages with ILogger in .NET Core. I showed how using string-interpolation can significantly hurt performance, and how using IsEnabled() and LoggerMessage.Define() can significantly improve performance, though at the expense of additional boilerplate code.

I then showed the new LoggerMessage source generator, described how to use it, and showed that the code it generates is very similar to the optimal LoggerMessage.Define() code, but without the boilerplate. I showed some of the extra features it has (static method generation, message template verification, dynamic log levels), and the impact those have on the generated code.

Finally, I benchmarked the various approaches, and showed again that the source generator comes out on top. If you're building an application on .NET 6, I strongly recommend using the source generator for your new code, and getting a free performance improvement!