In this post I provide a primer on the ASP.NET Core data-protection system: what it is, why do we need it, and how it works at a high level.
Why do we need the data-protection system?
The data-protection system is a set of cryptography APIs used by ASP.NET Core to encrypt data that must be handled by an untrusted third-party.
The classic example of this is authentication cookies. Cookies are a way of persisting state between requests. You don't want to have to provide your username and password with every request to a server, that would be very arduous!
Instead, you provide your credentials once to the server. The server verifies your details and issues a cookie that says "This is Andrew Lock. He doesn't need to provide any other credentials, trust him". On subsequent requests, you can simply provide that cookie, instead of having to supply credentials. Browsers automatically send cookies with subsequent requests, which is how the web achieves the smooth sign-in experience for users.
This cookie is a very sensitive item. Any request that includes the cookie will be treated as though it was sent by the original user, just as though they provided their username and password with every request. There are lots of protections in browsers (for example, CORS) to stop attackers getting access to these cookies.
However, it's not just a case of stopping others getting their hand on your cookies. As a website owner, you also don't want users to tamper with their own cookies.
Authentication cookies often contain more than just the ID or name of the user that authenticated. They typically contain a variety of additional claims. Claims are details about the user. They could be facts, such as their name, email, or phonenumber, but they can also be permission-related claims, such as IsAdmin or CanEditPosts.
If you're new to claims-based authentication, I wrote an introduction to authentication and claims several years ago that many people have found useful. Alternatively, see the authentication chapter in my book (shameless plug!).
These claims are typically needed for every request, to determine if the user is allowed to take an action. Instead of having to load these claims from a database with every request, they're typically included in the authentication cookie that's sent to the user.
This makes it easier for the application—once the user is authenticated by extracting the user principal from the cookie, the application will know exactly which claims the user has. That means the application has to trust the cookie. If the cookie was sent in plain-text, then the user could just edit the values, exposing a glaring security hole in the application.
The ASP.NET Core data-protection system is used for exactly this purpose. It encrypts and decrypts sensitive data such as the authentication cookie. By encrypting the authentication cookie before it's returned in the response, the application knows that the cookie has not been tampered with, and can trust its values.
How does the data-protection system work at a high level?
The data-protection system tries to solve a tricky problem: how to protect sensitive data that will be exposed to attackers, ideally without exposing any key material to developers, while following best practices for key-rotation and encryption at rest.
The data-protection system uses symmetric-key encryption to protect data. A key containing random data is used to encrypt the data, and the same key is used to decrypt the data.
The ASP.NET Core data-protection system assumes that it will be the same app or application decrypting the data as encrypted it. That implies it has access to the same key, and knows the parameters used to encrypt the data.
In a typical ASP.NET Core application there might be several different types of unrelated data you need to encrypt. For example, in addition to authentication cookies, you might also need to encrypt Cross-Site Request Forgery Tokens (CSRF) or password reset tokens.
You could use the same key for all these different purposes, but that has the potential for issues to creep in. It would be far better if a password reset token couldn't be "accidentally" (more likely, maliciously) used as an authentication token, for example.
The ASP.NET Core data-protection system achieves this goal by using "purposes". The data-protection system has a parent key which can't be used directly. Instead, you must derive child keys from the parent key, and it's those keys which are used to encrypt and decrypt the data.
Deriving a key from a parent key using the same purpose string will always give the same key material, so you can always decrypt data that was encrypted if you have the parent key and know the purpose string. If a key is derived using a different purpose, then attempting to decrypt the data will fail. That keeps the data isolated which is better for security.
The image above also shows that you can derive child keys from a child key. This can be useful in some multi-tenant scenarios for example. There's no special relationship between the child and grand-child keys—neither can read data encrypted with the other key. You can read the gory details about key derivation here.
In most cases you won't have to interact with the data-protection system directly to create keys or encrypt data. That's handled by the core ASP.NET Core framework and the accompanying libraries. They make sure to use unique strings for each different purpose in your application. You can create your own protectors and encrypt other data if you wish (see below), but that's not required for day-to-day running of ASP.NET Core applications.
I'm a .NET Framework developer—this sounds a lot like <machineKey>?
The data-protection system is new to ASP.NET Core, but the need to protect authentication tokens isn't new, so what were we using before? The answer is <machineKey>.
The <machineKey> element was used in much the same way as the ASP.NET Core data-protection system, to configure the keys and cryptography suite used to encrypt data by the authentication system (among other places). Unfortunately there were some complexities in using this key as it was typically read from the machine.config, would have to be configured on the machine running your application. When running in a cluster, you'd have to make sure to keep these keys in sync which could be problematic!
The need to keep keys in sync doesn't change with the data-protection system, it's just a lot easier to do, as you'll see shortly.
In .NET Framework 4.5 we got the ability to replace the <machineKey> element and the whole cryptography pipeline it uses. That means you can actually replace the default <machineKey> functionality with the new ASP.NET Core data-protection system, as long as you're running .NET Framework 4.5.1. You can read how to do that in the documentation.
Warning: If you're migrating ASP.NET applications to ASP.NET Core, and are sharing authentication cookies, you'll need to make sure you do this, so that authentication cookies can continue to be decrypted by all your applications.
I won't go any more into <machineKey> here, partly because that's the old approach, and partly because I don't know much about it! Needless to say, many of the challenges with managing the <machineKey> have been addressed in the newer data-protection system.
How is the data protection key managed? Do I need to rotate it manually?
If you know anything about security, you're probably used to hearing that you should regularly rotate passwords, secrets, and certificates. This can go some way to reducing the impact if one of your secrets is compromised. That's why HTTPS certificates are gradually being issued with smaller and smaller lifetimes.
How often the best-practice of secret-rotation is actually done is another question entirely. Depending on the support you get from your framework and tools, rotating secrets and certificates can be painful, especially in that transition period, where you may have to support both old and new secrets.
Given that the data-protection keys are critical for securing your ASP.NET Core applications, you won't be surprised that key rotation is the default for the data-protection system. By default, data-protection keys have a lifetime of 90 days, but you generally don't have to worry about that yourself. The data-protection system automatically creates new keys when old keys are near to expiration. The collection of all the available keys is called the key ring.
I won't go into the details of key management in this post. Just be aware that key rotation happens automatically, and as long as you don't delete any old keys (or explicitly revoke them), then encrypted data can still be retrieved using an expired key. Expired keys aren't used for encrypting new data.
Can I protect other data too or is it just for authentication cookies?
The data-protection system is used implicitly by ASP.NET Core to handle encryption and decryption of authentication tokens. It's also used by the ASP.NET Core Identity UI to protect password reset and MFA tokens. You don't need to do anything for this protection—the framework handles the protection itself.
If you have your own temporary data that you want to encrypt, you can use the data-protection APIs directly. I'll go into more detail in a later post, but the quick example below taken from the docs shows how you can use the IDataProtectionProvider service (registered by default in ASP.NET Core apps) to encrypt and decrypt some data:
public class MyClass
{
// The IDataProtectionProvider is registered by default in ASP.NET Core
readonly IDataProtectionProvider _rootProvider;
public MyClass(IDataProtectionProvider rootProvider)
{
_rootProvider = rootProvider;
}
public void RunSample()
{
// Create a child key using the purpose string
string purpose = "Contoso.MyClass.v1";
IDataProtector protector = provider.CreateProtector(purpose);
// Get the data to protect
Console.Write("Enter input: ");
string input = Console.ReadLine();
// Enter input: Hello world!
// protect the payload
string protectedPayload = _protector.Protect(input);
Console.WriteLine($"Protect returned: {protectedPayload}");
//PRINTS: Protect returned: CfDJ8ICcgQwZZhlAlTZT...OdfH66i1PnGmpCR5e441xQ
// unprotect the payload
string unprotectedPayload = _protector.Unprotect(protectedPayload);
Console.WriteLine($"Unprotect returned: {unprotectedPayload}");
//PRINTS: Unprotect returned: Hello world
}
}
Generally speaking though, this isn't something you'll want to do. I've personally only needed it when dealing with password reset and similar tokens, as mentioned previously.
Is there anything I shouldn't use data protection for?
An important point is that the data-protection system isn't really intended for general-purpose encryption. It's expected that you'll be encrypting things which, by their nature, have a limited lifetime, like authentication tokens and password reset tokens.
Warning: Don't use the data-protection system for long-term encryption. The data-protection keys are designed to expire and be rotated. Additionally, if keys are deleted (not recommended) then encrypted data will be permanently lost.
Theoretically, you could use the data-protection system for data that you wish to encrypt and store long-term, in a database for example. The data-protection keys expire every 90 days (by default), but you can still decrypt data with an expired key.
The real danger comes if the data-protection keys get deleted for some reason. This isn't recommended, but accidents happen. When used correctly, the impact of deleting data-protection keys on most applications would be relatively minor—users would have to log-in again, password reset keys previously issued would be invalid—annoying, but not a disaster.
If, on the other hand, you've encrypted sensitive data with the data-protection system and then stored that in the database, you have a big problem. That data is gone, destroyed. It's definitely not worth taking that risk! Instead you should probably use the dedicated encryption libraries in .NET Core, along with specific certificates or keys created for that purpose.
How do I configure data-protection in my ASP.NET Core application?
This is where the rubber really meets the road. Typically the only place you'll interact with the data-protection system is when you're configuring it, and if you don't configure it correctly you could expose yourself to security holes or not be able to decrypt your authentication cookies.
On the one hand, the data-protection system needs to be easy to configure and maintain, as complexity and maintenance overhead typically lead to bugs or poor practices. But ASP.NET Core also needs to run in a wide variety of environments: on Windows, Linux, macOS; in Azure, AWS, or on premises; on high end servers and a Raspberry Pi. Each of those platforms has different built-in cryptography mechanisms and features available, and .NET Core needs to be safe on all of them.
To work around this, the data-protection system uses a common "plugin" style architecture. There are basically two different pluggable areas:
- Key ring persistence Location: Where should the keys be stored?
- Persistence encryption: Should the keys be encrypted at rest, and if so how.
ASP.NET Core tries to to set these to sensible options by default. For example on a Windows (non-Azure App) machine, the keys will be persisted to %LOCALAPPDATA%\ASP.NET\DataProtection-Keys and encrypted at rest.
Unfortunately, most of the defaults won't work for you once you start running your application in production and scaling up your applications. Instead, you'll likely need to take a look at one of the alternative configuration approaches.
Summary
In this post I provided a high-level overview of ASP.NET Core's data protection system. I described the motivation for the data-protection system—transient, symmetric, encryption—and some of the design principles behind it. I described the system at a high level, with a master key that is used to derive child keys using "purpose" strings.
Next I described how the data-protection system is analogous to the <machineKey> in .NET Framework apps. In fact, there's a <machineKey> plugin, which allows you to use the ASP.NET Core data-protection system in your .NET Framework ASP.NET apps.
Finally, I discussed key rotation and persistence. This is a key feature of the data protection system, and is the main area on which you need to focus when configuring your application. Expired keys can be used to decrypt existing data, but they can't be used to encrypt new data. If you're running your application in a clustered scenario, you'll want to take a look at one of the alternative configuration approaches