Using strongly-typed entity IDs to avoid primitive obsession (Part 3)

In a previous post, I described a common problem in which primitive arguments (e.g. System.Guid or string) are passed in the wrong order to a method, resulting in bugs. This problem is a symptom of primitive obsession: using primitive types to represent higher-level concepts. In my second post, I showed how to create a JsonConverter and TypeConverter to make using the strongly-typed IDs easier with ASP.NET Core.

Martin Liversage noted that JSON.NET will use a TypeConverter where one exists, so you generally don't need the custom JsonConverter I provided in the previous post!

In this post, I discuss using strongly-typed IDs with EF Core. I personally don't use EF Core a huge amount, but after a little playing I came up with something that I thought worked pretty well. Unfortunately, there's one huge flaw which puts a cloud over the whole approach, as I'll describe later 🙁.

Interfacing with external system using strongly typed IDs

As a very quick reminder, strongly-typed IDs are types that can be used to represent the ID of an object, for example an OrderId or a UserId. A basic implementation (ignoring the overloads and converters etc.) looks something like this:

public readonly struct OrderId : IComparable<OrderId>, IEquatable<OrderId>
{
    public Guid Value { get; }
    public OrderId(Guid value)
    {
        Value = value;
    }

    // various helpers, overloads, overrides, implementations, and converters
}

You only get the full benefit of strongly-typed IDs if you can use them throughout your application. That includes at the "edges" of the app, where you interact with external systems. In the previous post I described the interaction at the public-facing end of your app, in ASP.NET Core MVC controllers.

The other main external system you will likely need to interface with is the database. At the end of the last post, I very briefly described a converter for using strongly-typed IDs with Dapper by creating a custom TypeHandler:

class OrderIdTypeHandler : SqlMapper.TypeHandler<OrderId>
{
    public override void SetValue(IDbDataParameter parameter, OrderId value)
    {
        parameter.Value = value.Value;
    }

    public override OrderId Parse(object value)
    {
        return new OrderId((Guid)value);
    }
}

This needs to be registered globally using SqlMapper.AddTypeHandler(new OrderIdTypeHandler()); to be used directly in your Dapper database queries.

Dapper is the ORM that I use the most in my day job, but EF Core is possibly going to be the most common ORM in ASP.NET Core apps. Making EF Core play nicely with the strongly-typed IDs is possible, but requires a bit more work.

Building an EF Core data model using strongly typed IDs

We'll start by creating a very simple data model that uses strongly-typed IDs. The classic ecommerce Order/OrderLine example contains everything we need:

public class Order
{
    public OrderId OrderId { get; set; }
    public string Name { get; set; }

    public ICollection<OrderLine> OrderLines { get; set; }
}

public class OrderLine
{
    public OrderId OrderId { get; set; }
    public OrderLineId OrderLineId { get; set; }
    public string ProductName { get; set; }
}

We have two entities:

• Order which has an OrderId, and has a collection of OrderLines.
• OrderLine which has as OrderLineId and an OrderId. All of the IDs are strongly-typed.

After creating these entities, we need to add them to the EF Core DbContext. We create a DbSet<Order> for the collection of Orders, and let EF Core discover the OrderLine entity itself:

 public class ApplicationDbContext : DbContext
{
    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
        : base(options)
    {
    }

    public DbSet<Order> Orders { get; set; }
}

Unfortunately, if we try and generate a new migration for updated model using the dotnet ef tool, we'll get an error:

> dotnet ef migrations add OrderSchema

System.InvalidOperationException: The property 'Order.OrderId' could not be mapped, 
because it is of type 'OrderId' which is not a supported primitive type or a valid 
entity type. Either explicitly map this property, or ignore it using the 
'[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.

EF Core complains that it doesn't know how to map our strongly-typed IDs (OrderId) to a database type. Luckily, there's a mechanism we can use to control this as of EF Core 2.1: value converters.

Creating a custom ValueConverter for EF Core

As described in the EF Core documentation:

Value converters allow property values to be converted when reading from or writing to the database. This conversion can be from one value to another of the same type (for example, encrypting strings) or from a value of one type to a value of another type (for example, converting enum values to and from strings in the database.)

The latter conversion, converting from one type to another, is what we need for the strongly-typed IDs. By using a value converter, we can convert our IDs into a Guid, just before they're written to the database. When reading a value, we convert the Guid value from the database into a strongly typed ID.

EF Core allows you to configure value converters manually for each property in your modelling code using lambdas. Alternatively, you can create reusable, standalone, custom value converters for each type. That's the approach I show here.

To implement a custom value converter you create an instance of a ValueConverter<TModel, TProvider>. TModel is the type being converted (the strongly-typed ID in our case), while TProvider is the database type. To create the converter you provide two lambda functions in the constructor arguments:

• convertToProviderExpression: an expression that is used to convert the strongly-typed ID to the database value (a Guid)
• convertFromProviderExpression: an expression that is used to convert the database value (a Guid) into an instance of the strongly-typed ID.

You can create an instance of the generic ValueConverter<> directly, but I chose to create a derived converter to simplify instantiating a new converter. Taking the OrderId example, we can create a custom ValueConverter<> using the following:

public class OrderIdValueConverter : ValueConverter<OrderId, Guid>
{
    public OrderIdValueConverter(ConverterMappingHints mappingHints = null)
        : base(
            id => id.Value,
            value => new OrderId(value),
            mappingHints
        ) { }
}

The lambda functions are simple - to obtain a Guid we use the Value property of the ID, and to create a new instance of the ID we pass the Guid to the constructor. The ConverterMappingHints parameter can allow setting things such as Scale and Precision for some database types. We don't need it here but I've included it for completeness in this example.

Registering the custom ValueConverter with EF Core's DB Context

The value converters describe how to store our strongly-typed IDs in the database, but EF Core need's to know when to use each converter. There's no way to do this using attributes, so you need to customise the model in DbContext.OnModelCreating. That makes for some pretty verbose code:

public class ApplicationDbContext : IdentityDbContext
{
    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
        : base(options)
    { }

    public DbSet<Order> Orders { get; set; }

    protected override void OnModelCreating(ModelBuilder builder)
    {
        base.OnModelCreating(builder);

        builder
            .Entity<Order>()
            .Property(o => o.OrderId)
            .HasConversion(new OrderIdValueConverter());

        builder
            .Entity<OrderLine>()
            .Property(o => o.OrderLineId)
            .HasConversion(new OrderLineIdValueConverter());

        builder
            .Entity<Order>()
            .Property(o => o.OrderId)
            .HasConversion(new OrderIdValueConverter());
    }
}

It's clearly not optimal having to add a manual mapping for each usage of a strongly-typed ID in your entities. Luckily we can simplify this code somewhat.

Automatically using a value converter for all properties of a given type.

Ideally our custom value converters would be used automatically by EF Core every time a given strongly-typed ID is used. There's an issue on GitHub for exactly this functionality, but in the meantime, we can emulate the behaviour by looping over all the model entities, as described in a comment on that issue.

In the code below, we loop over every entity in the model, and for each entity, find all those properties that are of the required type (OrderId for the OrderIdValueConverter). For each property we register the ValueConverter, in a process similar to the manual registrations above:

public static class ModelBuilderExtensions
{
    public static ModelBuilder UseValueConverter(
        this ModelBuilder modelBuilder, ValueConverter converter)
    {
        // The-strongly typed ID type
        var type = converter.ModelClrType;

        // For all entities in the data model
        foreach (var entityType in modelBuilder.Model.GetEntityTypes())
        {
            // Find the properties that are our strongly-typed ID
            var properties = entityType
                .ClrType
                .GetProperties()
                .Where(p => p.PropertyType == type);

            foreach (var property in properties)
            {
                // Use the value converter for the property
                modelBuilder
                    .Entity(entityType.Name)
                    .Property(property.Name)
                    .HasConversion(converter);
            }
        }

        return modelBuilder;
    }
}

All that remains is to register the value converter for each strongly-typed ID type in the DbContext:

public class ApplicationDbContext : IdentityDbContext
{
    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
        : base(options)
    { }

    public DbSet<Order> Orders { get; set; }

    protected override void OnModelCreating(ModelBuilder builder)
    {
        base.OnModelCreating(builder);

        builder.UseValueConverter(new OrderIdValueConverter())
        builder.UseValueConverter(new OrderLineIdValueConverter())
    }
}

It's a bit frustrating having to manually register each of these value converters - every time you create a new strongly typed ID you have to remember to register it in the DbContext.

Creating the ValueConverter implementation itself for every strongly-typed ID is not a big deal if you're using snippets to generate your IDs, like I described in the last post.

It would be nice if we were able to generate a new ID, use it in an entity, and not have to remember to update the OnModelCreating method.

Automatically registering value converters for strongly typed IDs

We can achieve this functionality with a little bit of reflection and some attributes. We'll start by creating an attribute that we can use to link each strongly-typed ID to a specific value converter, called EfCoreValueConverterAttribute:

public class EfCoreValueConverterAttribute : Attribute
{
    public EfCoreValueConverterAttribute(Type valueConverter)
    {
        ValueConverter = valueConverter;
    }

    public Type ValueConverter { get; }
}

We'll decorate each strongly typed ID with the attribute as part of the snippet generation, which will give something like the following:

// The attribute links the OrderId to OrderIdValueConverter
[EfCoreValueConverter(typeod(OrderIdValueConverter))]
public readonly struct OrderId : IComparable<OrderId>, IEquatable<OrderId>
{
    public Guid Value { get; }
    public OrderId(Guid value)
    {
        Value = value;
    }

    // The ValueConverter implementation
    public class OrderIdValueConverter : ValueConverter<OrderId, Guid>
    {
        public OrderIdValueConverter()
            : base(
                id => id.Value,
                value => new OrderId(value)
            ) { }
    }
}

Next, we'll add another method to the ModelBuilderExtensions this loops through all the types in an Assembly and finds all those that are decorated with the EfCoreValueConverterAttribute (i.e. the strongly typed IDs). The Type of the value converter is extracted from the custom attribute, and an instance of the value converter is created using the ValueConverter. We can then pass that to the UseValueConverter method we created previously.

public static class ModelBuilderExtensions
    {
        public static ModelBuilder AddStronglyTypedIdValueConverters<T>(
            this ModelBuilder modelBuilder)
        {
            var assembly = typeof(T).Assembly;
            foreach (var type in assembly.GetTypes())
            {
                // Try and get the attribute
                var attribute = type
                    .GetCustomAttributes<EfCoreValueConverterAttribute>()
                    .FirstOrDefault();

                if (attribute is null)
                {
                    continue;
                }

                // The ValueConverter must have a parameterless constructor
                var converter = (ValueConverter) Activator.CreateInstance(attribute.ValueConverter);

                // Register the value converter for all EF Core properties that use the ID
                modelBuilder.UseValueConverter(converter);
            }

            return modelBuilder;
        }

        // This method is the same as shown previously
        public static ModelBuilder UseValueConverter(
            this ModelBuilder modelBuilder, ValueConverter converter)
        {
            var type = converter.ModelClrType;

            foreach (var entityType in modelBuilder.Model.GetEntityTypes())
            {
                var properties = entityType
                    .ClrType
                    .GetProperties()
                    .Where(p => p.PropertyType == type);

                foreach (var property in properties)
                {
                    modelBuilder
                        .Entity(entityType.Name)
                        .Property(property.Name)
                        .HasConversion(converter);
                }
            }

            return modelBuilder;
        }
    }

With this code in place, we can register all our value converters in one fell swoop in the DbContext.OnModelCreating method:

public class ApplicationDbContext : IdentityDbContext
{
    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
        : base(options)
    {
    }

    public DbSet<Order> Orders { get; set; }

    protected override void OnModelCreating(ModelBuilder builder)
    {
        base.OnModelCreating(builder);

        // add all value converters
        builder.AddStronglyTypedIdValueConverters<OrderId>();
    }
}

The type parameter OrderId in the above example is used to identify the Assembly to scan to find the strongly-typed IDs. If required, it would be simple to add another overload to allowing scanning multiple assemblies.

With the code above, we don't have to touch the DbContext when we add a new strongly-typed ID, which is a much better experience for developers. If we run the migrations now, all is well:

> dotnet ef migrations add OrderSchema

Done. To undo this action, use 'ef migrations remove'

If you check the generated migration, you'll see that the OrderId column is created as a non-nullable Guid, and is the primary key, as you'd expect.

public partial class OrderSchema : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.CreateTable(
            name: "Orders",
            columns: table => new
            {
                OrderId = table.Column<Guid>(nullable: false),
                Name = table.Column<string>(nullable: true)
            },
            constraints: table =>
            {
                table.PrimaryKey("PK_Orders", x => x.OrderId);
            });
    }
}

This solves most of the problems you'll encounter using strongly typed IDs with EF Core, but there's one place where this doesn't quite work, and unfortunately, it might be a deal breaker.

Custom value converters result in client-side evaluation

Saving entities that use your strongly-typed IDs to the database is no problem for EF Core. However, if you try and load an entity from the database, and filter based on the strongly-typed ID:

var order = _dbContext.Orders
    .Where(order => order.OrderId == orderId)
    .FirstOrDefault();

then you'll see a warning in the logs that the where clause must be evaluated client-side:

warn: Microsoft.EntityFrameworkCore.Query[20500]
      The LINQ expression 'where ([x].OrderId == __orderId_0)' 
      could not be translated and will be evaluated locally.

info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (12ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      SELECT [x].[OrderId], [x].[Name]
      FROM [Orders] AS [x]

That's terrible. This query has got to be a contender for the most common thing you'll ever do, and the above solution will not be good enough. Fetching an Order by ID with client-side execution involves loading all Orders into memory and filtering in memory!

In fairness the documentation does mention this limitation right at the bottom of the page (emphasis mine):

Use of value conversions may impact the ability of EF Core to translate expressions to SQL. A warning will be logged for such cases. Removal of these limitations is being considered for a future release.

But this value converter is pretty much the most basic you could imagine - if this converter results in client-side evaluation, they all will!

There is an issue to track this problem, but unfortunately there's no easy work around to this one. 🙁

All is not entirely lost. It's not pretty, but after some playing I eventually found something that will let you use strongly-typed IDs in your EF Core models that doesn't force client-side evaluation.

Avoiding client-side evaluation in EF Core with conversion operators

The key is adding implicit or explicit conversion operators to the strongly-typed IDs, such that EF Core doesn't bork on seeing the strongly-typed ID in a query. There's two possible options, an explicit conversion operator, or an implicit conversion operator.

Using an explicit conversion operator with strongly typed IDs

The first approach is to add an explicit conversion operator to your strongly-typed ID to go from the ID type to a Guid:

public readonly struct OrderId
{
    public static explicit operator Guid(OrderId orderId) => orderId.Value;

    // Remainder of OrderId implementation ...
}

Adding this sort of operator means you can cast an OrderId to a Guid, for example:

var orderId = new OrderId(Guid.NewGuid());
var result = (Guid) orderId; // Only compiles with explicit operator

So how does that help? Essentially we can trick EF Core into running the query server-side, by using a construction similar to the following:

Guid orderIdValue = orderId.Value; // extracted for clarity, can be inlined
var order = _dbContext.Orders
    .Where(order => (Guid) order.OrderId == orderIdValue) // explicit conversion to Guid
    .FirstOrDefault();

The key point is the explicit conversion of order.OrderId to a Guid. When EF Core evaluates the query, it no longer sees an OrderId type that it doesn't know what to do with, and instead generates the SQL we wanted in the first place:

info: Microsoft.EntityFrameworkCore.Database.Command[20101]
      Executed DbCommand (7ms) [Parameters=[@__orderId_Value_0='?' (DbType = Guid)], CommandType='Text', CommandTimeout='30']
      SELECT TOP(1) [x].[OrderId], [x].[Name]
      FROM [Orders] AS [x]
      WHERE [x].[OrderId] = @__orderId_Value_0

This shows the where clause being sent to the database, so all is well again. Well, apart from the fact it's an ugly hack. 😕 Implicit operators make the process very slightly less ugly.

Using an implicit conversion operator with strongly typed IDs

The implicit conversion operator implementation is almost identical to the explicit implementation, just with a different keyword:

public readonly struct OrderId
{
    public static implicit operator Guid(OrderId orderId) => orderId.Value;

    // Remainder of OrderId implementation ...
}

With this code, you no longer need an explicit (Guid) cast to convert an OrderId to a Guid, so we can write the query as:

Guid orderIdValue = orderId.Value; // extracted for clarity, can be inlined
var order = _dbContext.Orders
    .Where(order => order.OrderId == orderIdValue) // Implicit conversion to Guid
    .FirstOrDefault();

This query generates identical SQL, so technically you could use either approach. But which should you choose?

Implicit vs Explicit operators

For simple ugliness, the implicit operator seems slightly preferable, as you don't have to add the extra cast, but I'm not sure if that's a bad thing. The trouble is that the implicit conversions apply throughout your code base, so suddenly code like this will compile:

public Order GetOrderForUser(Guid orderId, Guid userId)
{
    // Get the User
}

OrderId orderId = OrderId.New();
UserId userId = UserId.New();

var order = GetOrderForUser(userId, orderId); // arguments reversed, the bug is back!

The GetOrderForUser() method should obviously be using the strongly-typed IDs, but the fact that this is possible without any indication of errors just makes me a little uneasy. For that reason, I think I prefer the explicit operators.

Either way, you should definitely hide away the cast from callers wherever possible:

// with explicit operator
public class OrderService
{
    // public API uses strongly-typed ID
    public Order GetOrder(OrderId orderId) => GetOrder(orderId.Value);

    // private implementation to handle casting
    private Order GetOrder(Guid orderId)
    {
        return _dbContext.Orders
            .Where(x => (Guid) x.OrderId == orderId)
            .FirstOrDefault();
    }
}

// with implicit operator 
public class OrderService
{
    // public API uses strongly-typed ID
    public Order GetOrder(OrderId orderId) => GetOrder(orderId.Value);

    // private implementation to handle casting
    private Order GetOrder(Guid orderId)
    {
        return _dbContext.Orders
            .Where(x => x.OrderId == orderId) // Only change is no cast required here
            .FirstOrDefault();
    }
}

It's probably also worth configuring your DbContext to throw an error when client-side evaluation occurs so you don't get client-side errors creeping in without you noticing. Override the DbContext.OnConfiguring method, and configure the options:

protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
    optionsBuilder.ConfigureWarnings(warning => 
            warning.Throw(RelationalEventId.QueryClientEvaluationWarning));
}

Even with all this effort, there's still gotchas. As well as standard IQueryable<T> LINQ syntax, the DbSet<> exposes a Find method which is effectively a shorthand for SingleOrDefault() for querying by an entities primary key. Unfortunately, nothing we do here will work:

var orderId = new OrderId(Guid.NewGuid());
_dbContext.Find(orderId); // using the strongly typed ID directly causes client-side evaluations
_dbContext.Find(order.Value); // passing in a Guid causes an exception : The key value at position 0 of the call to 'DbSet<Order>.Find' was of type 'Guid', which does not match the property type of 'OrderId'.

So close…

This post is plenty long enough, and I haven't quite worked out a final solution but I have a couple of ideas. Check back in a couple of days, and hopefully I'll have it figured out 🙂

Summary

In this post I explored possible solutions that would allow you to use strongly-typed IDs directly in your EF Core entities. The ValueConverter approach described in this post gets you 90% of the way there, but unfortunately the fact that queries will be executed client-side really makes the whole approach more difficult until this issue is resolved. You can get some success by using explicit or implicit conversions, but there are still edge cases. I'm playing with a different approach as we speak, and hope to have something working in a couple of days, so check back soon!