ASP.NET Core includes a logging abstraction that makes writing logs to multiple locations easy. All of the first-party libraries that make up ASP.NET Core and EF Core use this abstraction, and the vast majority of libraries written for ASP.NET Core will too. That means its easy to aggregate the logs from your entire app, including the framework and your own code, into one place.
In this post I'll show how to create a logging provider that writes logs to the file system. In production, I'd recommended using a more fully-featured system like Serilog instead of this library, but I wanted to see what was involved to get a better idea of the process myself.
The code for the file logging provider is available on GitHub, or as the NetEscapades.Extensions.Logging.RollingFile package on NuGet.
The ASP.NET Core logging infrastructure
The ASP.NET Core logging infrastructure consists of three main components:
ILogger- Used by your app to create log messages.ILoggerFactory- Creates instances ofILoggerILoggerProvider- Controls where log messages are output. You can have multiple logging providers - every log message you write to anILoggeris written to the output locations for every configured logging provider in your app.
When you want to write a log message in your application you typically use DI to inject an ILogger<T> into the class, where T is the name of the class. The T is used to control the category associated with the class.
For example, to write a log message in an ASP.NET Core controller, HomeController, you would inject the ILogger<HomeController> and call one of the logging extension methods on ILogger:
public class HomeController: Controller
{
private readonly ILogger<HomeController> _logger;
public HomeController(ILogger<HomeController> logger)
{
_logger = logger;
}
public IActionResult Get()
{
_logger.LogInformation("Calling home controller action");
return View();
}
}
This will write a log message to each output of the configured logging providers, something like this (for the console logger):
info: ExampleApplication.Controllers.HomeController[0]
Calling home controller action
ASP.NET Core includes several logging providers out of the box, which you can use to write your log messages to various locations:
- Console provider - writes messages to the Console
- Debug provider - writes messages to the Debug window (e.g. when debugging in Visual Studio)
- EventSource provider - writes messages using Event Tracing for Windows (ETW)
- EventLog provider - writes messages to the Windows Event Log
- TraceSource provider - writes messages using System.Diagnostics.TraceSource libraries
- Azure App Service provider - writes messages to blob storage or files when running your app in Azure.
In ASP.NET Core 2.0, the console and Debug loggers are configured by default, but in production you'll probably want to write your logs to somewhere more durable. In modern applications, you'll likely want to write to a centralised location, such as an Elastic Search cluster, Seq, elmah.io, or Loggr.
You can write your logs to most of these locations by adding logging providers for them directly to your application, but one provider is particularly conspicuous by its absence - a file provider. In this post I'll show how to implement a logging provider that writes your application logs to rolling files.
The logging library Serilog includes support for logging to files, as well as a multitude of other sinks. Rather than implementing your own logging provider as I have here, I strongly recommend you check it out. Nicholas Blumhardt has a post on adding Serilog to your ASP.NET Core 2.0 application here.
Creating a rolling file based logging provider
In actual fact, the ASP.NET Core framework does include a file logging provider, but it's wrapped up behind the Azure App Service provider. To create the file provider I mostly used files already part of the Microsoft.Extensions.Logging.AzureAppServices package, and exposed it as a logging provider in it's own right. A bit of a cheat, but hey, "shoulders of giants" and all that.
Implementing a logging provider basically involves implementing two interfaces:
ILoggerILoggerProvider
The AzureAppServices library includes some base classes for batching log messages up, and writing them on a background thread. That's important as logging should inherently be a quick and synchronous operation. Your app shouldn't know or care where the logs are being written, and it certainly shouldn't be waiting on file IO!
The batching logger provider
The BatchingLoggerProvider is an abstract class that encapsulates the process of writing logs to a concurrent collection and writing them on a background thread. The full source is here but the abridged version looks something like this:
public abstract class BatchingLoggerProvider : ILoggerProvider
{
protected BatchingLoggerProvider(IOptions<BatchingLoggerOptions> options)
{
// save options etc
_interval = options.Value.Interval
// start the background task
_outputTask = Task.Factory.StartNew<Task>(
ProcessLogQueue,
null,
TaskCreationOptions.LongRunning);
}
// Implemented in derived classes to actually write the messages out
protected abstract Task WriteMessagesAsync(IEnumerable<LogMessage> messages, CancellationToken token);
// Take messages from concurrent queue and write them out
private async Task ProcessLogQueue(object state)
{
while (!_cancellationTokenSource.IsCancellationRequested)
{
// Add pending messages to the current batch
while (_messageQueue.TryTake(out var message))
{
_currentBatch.Add(message);
}
// Write the current batch out
await WriteMessagesAsync(_currentBatch, _cancellationTokenSource.Token);
_currentBatch.Clear();
// Wait before writing the next batch
await Task.Delay(interval, cancellationToken);
}
}
// Add a message to the concurrent queue
internal void AddMessage(DateTimeOffset timestamp, string message)
{
if (!_messageQueue.IsAddingCompleted)
{
_messageQueue.Add(new LogMessage { Message = message, Timestamp = timestamp }, _cancellationTokenSource.Token);
}
}
public void Dispose()
{
// Finish writing messages out etc
}
// Create an instance of an ILogger, which is used to actually write the logs
public ILogger CreateLogger(string categoryName)
{
return new BatchingLogger(this, categoryName);
}
private readonly List<LogMessage> _currentBatch = new List<LogMessage>();
private readonly TimeSpan _interval;
private BlockingCollection<LogMessage> _messageQueue = new BlockingCollection<LogMessage>(new ConcurrentQueue<LogMessage>());
private Task _outputTask;
private CancellationTokenSource _cancellationTokenSource = new CancellationTokenSource();
}
The BatchingLoggerProvider starts by creating a Task on a background thread that runs the ProcessLogQueue method. This method sits in a loop until the provider is disposed and the CancellationTokenSource is cancelled. It takes log messages off the concurrent (thread safe) queue, and adds them to a temporary list, _currentBatch. This list is passed to the abstract WriteMessagesAsync method, implemented by derived classes, which writes the actual logs to the destination.
The other most important method is CreateLogger(categoryName), which creates an instance of an ILogger that is injected into your classes. Our actual non-abstract provider implementation, the FileLoggerProvider, derives from the BatchingLoggerProvider:
[ProviderAlias("File")]
public class FileLoggerProvider : BatchingLoggerProvider
{
private readonly string _path;
private readonly string _fileName;
private readonly int? _maxFileSize;
private readonly int? _maxRetainedFiles;
public FileLoggerProvider(IOptions<FileLoggerOptions> options) : base(options)
{
var loggerOptions = options.Value;
_path = loggerOptions.LogDirectory;
_fileName = loggerOptions.FileName;
_maxFileSize = loggerOptions.FileSizeLimit;
_maxRetainedFiles = loggerOptions.RetainedFileCountLimit;
}
// Write the provided messages to the file system
protected override async Task WriteMessagesAsync(IEnumerable<LogMessage> messages, CancellationToken cancellationToken)
{
Directory.CreateDirectory(_path);
// Group messages by log date
foreach (var group in messages.GroupBy(GetGrouping))
{
var fullName = GetFullName(group.Key);
var fileInfo = new FileInfo(fullName);
// If we've exceeded the max file size, don't write any logs
if (_maxFileSize > 0 && fileInfo.Exists && fileInfo.Length > _maxFileSize)
{
return;
}
// Write the log messages to the file
using (var streamWriter = File.AppendText(fullName))
{
foreach (var item in group)
{
await streamWriter.WriteAsync(item.Message);
}
}
}
RollFiles();
}
// Get the file name
private string GetFullName((int Year, int Month, int Day) group)
{
return Path.Combine(_path, $"{_fileName}{group.Year:0000}{group.Month:00}{group.Day:00}.txt");
}
private (int Year, int Month, int Day) GetGrouping(LogMessage message)
{
return (message.Timestamp.Year, message.Timestamp.Month, message.Timestamp.Day);
}
// Delete files if we have too many
protected void RollFiles()
{
if (_maxRetainedFiles > 0)
{
var files = new DirectoryInfo(_path)
.GetFiles(_fileName + "*")
.OrderByDescending(f => f.Name)
.Skip(_maxRetainedFiles.Value);
foreach (var item in files)
{
item.Delete();
}
}
}
}
The FileLoggerProvider implements the WriteMessagesAsync method by writing the log messages to the file system. Files are created with a standard format, so a new file is created every day. Only the last _maxRetainedFiles files are retained, as defined by the FileLoggerOptions.RetainedFileCountLimit property set on the IOptions<> object provided in the constructor.
Note In this implementation, once files exceed a maximum size, no further logs are written for that day. The default is set to 10MB, but you can change this on the
FileLoggerOptionsobject.
The [ProviderAlias("File")] attribute defines the alias for the logger that you can use to configure log filtering. You can read more about log filtering in the docs.
The FileLoggerProvider is used by the ILoggerFactory to create an instance of the BatchingLogger, which implements ILogger, and is used to actually write the log messages.
The batching logger
The BatchingLogger is pretty simple. The main method, Log, passes messages to the provider by calling AddMessage. The methods you typically use in your app, such as LogError and LogInformation are actually just extension methods that call down to this underlying Log method.
public class BatchingLogger : ILogger
{
private readonly BatchingLoggerProvider _provider;
private readonly string _category;
public BatchingLogger(BatchingLoggerProvider loggerProvider, string categoryName)
{
_provider = loggerProvider;
_category = categoryName;
}
public IDisposable BeginScope<TState>(TState state)
{
return null;
}
public bool IsEnabled(LogLevel logLevel)
{
return logLevel != LogLevel.None;
}
// Write a log message
public void Log<TState>(DateTimeOffset timestamp, LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
if (!IsEnabled(logLevel))
{
return;
}
var builder = new StringBuilder();
builder.Append(timestamp.ToString("yyyy-MM-dd HH:mm:ss.fff zzz"));
builder.Append(" [");
builder.Append(logLevel.ToString());
builder.Append("] ");
builder.Append(_category);
builder.Append(": ");
builder.AppendLine(formatter(state, exception));
if (exception != null)
{
builder.AppendLine(exception.ToString());
}
_provider.AddMessage(timestamp, builder.ToString());
}
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception, Func<TState, Exception, string> formatter)
{
Log(DateTimeOffset.Now, logLevel, eventId, state, exception, formatter);
}
}
Hopefully this class is pretty self explanatory - most of the work is done in the logger provider.
The remaining piece of the puzzle is to provide the extension methods that let you easily configure the provider for your own app.
Extension methods to add the provider to your application
In ASP.NET Core 2.0, logging providers are added to your application by adding them directly to the WebHostBuilder in Program.cs. This is typically done using extension methods on the ILoggingBuilder. We can create a simple extension method, and even add an override to allow configuring the logging provider's options (filenames, intervals, file size limits etc).
public static class FileLoggerFactoryExtensions
{
public static ILoggingBuilder AddFile(this ILoggingBuilder builder)
{
builder.Services.AddSingleton<ILoggerProvider, FileLoggerProvider>();
return builder;
}
public static ILoggingBuilder AddFile(this ILoggingBuilder builder, Action<FileLoggerOptions> configure)
{
builder.AddFile();
builder.Services.Configure(configure);
return builder;
}
}
In ASP.NET Core 2.0, logging providers are added using DI, so adding our new logging provider just requires adding the FileLoggerProvider to DI, as in the AddFile() method above.
With the provider complete, we can add it to our application:
public class Program
{
public static void Main(string[] args)
{
BuildWebHost(args).Run();
}
public static IWebHost BuildWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureLogging(builder => builder.AddFile()) // <- add this line
.UseStartup<Startup>()
.Build();
}
This adds the FileLoggerProvider to the application, in addition to the Console and Debug provider. Now when we write logs to our application, logs will also be written to a file:
Summary
Creating an ILoggerProvider will rarely be necessary, especially thanks to established frameworks like Serilog and NLog that integrate with ASP.NET Core. Wherever possible, I suggest looking at one of these, but if you don't want to use a replacement framework like this, then using a dedicated ILoggerProvider is an option.
Implementing a new logging provider requires creating an ILogger implementation and an ILoggerProvider implementation. In this post I showed an example of a rolling file provider. For the full details and source code, check out the project on GitHub, or the NuGet package. All comments, bugs and suggestions welcome, and credit to the ASP.NET team for creating the code I based this on!