Exploring the new rollForward and allowPrerelease settings in global.json: Exploring ASP.NET Core 3.0 - Part 8

I was listening to the Azure DevOps Podcast with Jeffrey Palermero recently and heard Kathleen Dollard mention that there were some updates to the global.json file and .NET Core SDK in 3.0. This post explores those additions and the effects they have on SDK selection for a machine with multiple SDKs installed.

The official documentation for this behaviour covers everything in this post, but I found it pretty hard to internalise the various rules it describes. This post primarily adds some extra background, a couple of pictures, and explores the rules using examples to make it easier to grok!

The .NET Core Runtime vs the .NET Core SDK.

Before we start, it's important to understand the difference between the .NET Code runtime and the .NET Core SDK:

• The .NET Core runtime is what runs a .NET application. It has very limited functionality - it literally just runs a compiled application. It's the important piece when you're running your application in production.
• The .NET Core SDK does everything else: it compiles your application, tests it, downloads NuGet packages, and a whole lot more. This is the important piece when you're developing your application.

Generally speaking, you need to choose the version of the .NET Core runtime you use carefully. Different versions have different support windows (depending if they're LTS or current) and have different features. It's the runtime version that you specify in the <TargetFramework> element of your project file. For example, the project file below specifies that the .NET Core 3.1 runtime should be used:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>

</Project>

In contrast, you generally don't specify a version of the .NET Core SDK that's needed to build the application. Normally all that matters is that you have a version of the SDK that supports the given runtime version. So to target the 3.1 runtime, you'll need an SDK version that supports building for it, e.g. version SDK 3.1.101 of the SDK.

An important thing to note, is that the .NET Core SDK is supposed to be backwards compatible. So the 3.1.101 SDK can build .NET Core 2.1 applications and 2.2 applications etc as well. In other words, you generally don't need to use a specific version of the .NET Core SDK. Any SDK that is high enough will do.

Specifying a specific SDK version with a global.json

Typically then you shouldn't have to worry about which versions of the SDK you have installed. Backwards compatibility means that the most recent SDK should be able to build everything.

Unfortunately that's not always the case: bugs creep in, features change, and sometimes you need (or want) a specific version of the SDK installed. Some peripheral things change from version-t0-version too, such as the project templates that you use with dotnet new. The templates that come with the .NET Core 1.0 SDK are very different to those that come with the .NET Core 3.1 SDK for example.

For that reason, sometimes you might want to "pin" the version of the .NET Core SDK to a specific version for a particular project or for a particular folder. For example you might want one specific project to use the .NET Core 1.0 SDK, while letting all the other projects on your machine use the latest 3.1 version. To do that, you can use a global.json file.

Whenever you run a dotnet SDK command like dotnet build, dotnet publish, or dotnet new, the dotnet.exe entrypoint looks for a global.json file in the same directory as the command being run. If it doesn't find one, it looks in the parent directory instead. If it still doesn't find one it keeps working up through the parent directories until it finds a global.json file, or until it reaches the root directory.

At this point dotnet.exe will either have the "nearest" global.json file, or no file at all. It then uses the values in the global.json file (or the absence of the file), to decide which SDK version to use to handle the command (dotnet build etc).

The rules governing which version to use depends on four things:

• Which versions of the .NET Core SDK do you have installed?
• Which version does the global.json request (if any)
• What is the current "roll-forward" policy for SDK versions
• Are pre-release versions allowed to be used?

We'll look at how each of those affect the final version of the SDK selected in the remainder of the post.

How to see which versions of the .NET Core SDK you have installed

The first variable for determining which version of the .NET Core SDK will be used to run a command, is which SDKs are available. Thankfully that's easy to check in .NET Core 3+, as you can use dotnet --list-sdks.

Running the command on my machine, I get:

> dotnet --list-sdks
1.1.14 [C:\Program Files\dotnet\sdk]
2.1.600 [C:\Program Files\dotnet\sdk]
2.1.602 [C:\Program Files\dotnet\sdk]
2.1.604 [C:\Program Files\dotnet\sdk]
2.1.700 [C:\Program Files\dotnet\sdk]
2.1.801 [C:\Program Files\dotnet\sdk]
2.2.203 [C:\Program Files\dotnet\sdk]
3.0.100 [C:\Program Files\dotnet\sdk]
3.1.101 [C:\Program Files\dotnet\sdk]

This shows that I have 9 SDKs currently installed (all in C:\Program Files\dotnet\sdk).

Understanding the .NET Core SDK version numbers

A slightly tricky aspect of the .NET Core SDK, is that it doesn't really use the semantic versioning that you may be familiar with, and which the runtime uses. It kind of does, but it's a bit more complicated than that, (plus it's changed throughout the last few versions of .NET Core).

Currently, the .NET Core SDK version number is broken down into 4 sections. For example, for the 2.1.801 SDK version:

Those different sections will become important in the next section, when we look at "roll-forward" policies. Broadly speaking, the major and minor version numbers align with the major and minor version of .NET Core, e.g. 2.2 or 3.1. The feature version is the complicated one, where it gets incremented when new features are added to the SDK (potentially without any changes in the runtime). The patch version is for patches to a given feature version.

The global.json schema in .NET Core 1/2 is very limited

The global.json file has been available since .NET Core 1.0, and up until the recent changes in .NET Core 3.0, it had a very simple structure:

{
  "sdk": {
    "version": "2.1.600"
  }
}

The version in the global.json would define which version of the SDK you needed. If it was installed, that version of the SDK would be used, otherwise you would get an error message similar to the following:

A compatible installed .NET Core SDK for global.json version [2.1.600] from [C:\repos\globaljsontest\global.json] was not found
Install the [2.1.600] .NET Core SDK or update [C:\repos\globaljsontest\global.json] with an installed .NET Core SDK

To be precise, the legacy behaviour was to use the patch roll-forward policy which we'll discuss shortly.

Specifying a single version was often rather limiting. In many cases the intention of the version number was used to indicate either a minimum SDK that was needed, or alternatively a maximum major version. Unfortunately the version field does not support wildcards, so that wasn't possible, and proved a poor substitute.

For example, if you still had projects stuck on .NET Core 1.0, you might add a global.json to require a 1.x SDK, like 1.1.14. In reality, you likely wouldn't need a specific version of the 1.x SDK (1.0.1 or 1.1.13 would work just fine), but the global.json forces you to specify a single version.

That's a pain, as it forces anyone using your project to install a new SDK version, when they may well have one that works just fine already.

Additions to global.json in .NET Core 3.0

In NET Core 3.0, the global.json file got a couple of important updates, the rollForward and allowPrerelease fields:

{
  "sdk": {
    "version": "2.1.600",
    "allowPrerelease": true,
    "rollForward": "patch"
  }
}

The algorithm for determining which version of the SDK to use was relatively simple in .NET Core 1.x/2.x - if a global.json was found and the requested SDK version was installed, that version (or a patched version) was used.

In .NET Core 3.0 the algorithm gets rather more complex, giving you extra controls. The flow chart below shows how values for the version, allowPrerelease, and rollForward values are determined based on the presence of the global.json, whether each field is present in the global.json, and whether or not the command is being run explicitly from the command line, or it's being run implicitly by Visual Studio (VS) (for example when Visual Studio runs a build):

At the end of this flow chart, we have values for the following:

• version: Either a specific version requested by a global.json, or if none was set, the highest installed version.
• allowPrerelease: Determines whether prerelease/preview SDKs that are installed should be considered when calculating which SDK version to use (e.g. 3.1.100-preview1)
• rollForward: The roll-forward policy to apply.

This brings us to the most complex section, understanding the roll-forward policy, and how each option controls which version of the SDK is selected.

Understanding the various rollForward policies

The roll-forward policy is used to determine which of the various installed SDKs should be selected when a given version is requested. By changing the roll-forward policy, you can relax or tighten the selection criteria. That's a bit vague, but we'll looks at some examples soon.

In .NET Core 3.x, there are now nine different values for the rollForward policy, which can broadly be separated into three different categories:

First we have the disable policy:

• disable: If the requested version doesn't exist, then fail outright. Don't ever use an SDK version other than the specific version requested.

Next we have the conservative roll-forward policies, which get progressively more lenient in looking for suitable SDK versions:

• patch: If the requested version doesn't exist, use the highest installed SDK version with the same major, minor, and feature value e.g. 2.1.6xx
• feature: Use the highest installed SDK version with the same major, minor, and feature value e.g. 2.1.6xx. If no such version exists, uses the next installed SDK version with the same major and minor value e.g. 2.1.7xx, otherwise fails.
• minor: Apply the feature policy. If no suitable SDK version is found, use the highest installed SDK version with the same major value e.g. 2.x.xxx
• major: Apply the minor policy. If no suitable SDK version is found, use the highest installed SDK version, e.g. x.x.xxx

Finally we have the "latest" roll-forward policies, which always try and use the latest versions of suitable SDKs:

• latestPatch : Always use the highest installed SDK version with the same major, minor, and feature value e.g. 2.1.6xx
• latestFeature : Uses highest installed SDK version with the same major and minor value e.g. 2.1.xxx
• latestMinor : Uses highest installed SDK version with the same major value e.g. 2.x.xxx
• latestMajor : Uses highest installed SDK version

I'm aware that's a lot of information to digest! The conservative policies in particular are quite confusing, as the patch policy works subtly differently to the others. I think it's easiest to understand the differences by looking at examples, so the next few sections run through various scenarios, and describe the results in each case.

In each of these scenarios, I'm building a project on a system with the following SDKs installed (listed using dotnet --list-sdks):

1.1.14
2.1.600
2.1.602
2.1.604
2.1.700
2.1.801
2.2.203
3.0.100
3.1.101

You can view the SDK version that was selected based on a given global.json by running dotnet --version in the same folder. When there is no global.json in the folder (or in any parent folders) you should see the latest SDK installed on your machine:

> dotnet --version
3.1.101

Note that I'm ignoring the allowPrerelease flag in these tests. It doesn't have any impact if you don't have preview SDK versions installed. If you do have preview SDKs installed, the results will follow the same patterns shown below.

When the requested SDK version is available

Lets start first by selecting an SDK version that does exist on the system, 2.1.600, and see which SDK version is selected for all the different rollForward values. I create a global.json (by running dotnet new global.json) and change the rollForward property to test each policy:

{
  "sdk": {
    "version": "2.1.600",
    "rollForward": "xxx"
  }
}

Running dotnet --version after applying each of the roll-forward policies in turn gives the following results:

Note that even though we have the exact requested version available, 2.1.600, only the disable and patch policies use the actual SDK. Everything else uses at least a patched version of the SDK. Also note that the "conservative" policies, only use the patched version, even though we have additional minor and major versions available.

When the requested SDK version is not available

Now lets try requesting an SDK version that doesn't exist on our machine, 2.1.601. Other than that, we have the same global.json and the same SDKs installed.

{
  "sdk": {
    "version": "2.1.601",
    "rollForward": "xxx"
  }
}

Running dotnet --version after applying each of the roll-forward policies in turn gives the following results:

The results are almost identical to the previous case, with the following exceptions:

• The disable policy causes all SDK commands to fail.
• The patch policy skips the next-highest patch version, 2.1.602, and uses the latest patch 2.1.604 instead.

When no higher patch version exists

Finally, let's imagine that we've requested the 2.1.605 SDK, which has a higher patch version than any of the 2.1.6xx SDKs installed on the machine. Let's see what happens in this case:

Now we have some interesting results:

• With no high enough SDK versions matching the 2.1.6xx pattern the disable, patch, and latestPatch policies all fail.
• The other latest* policies use the same versions they have in all other experiments.
• The conservative policies (feature, minor, and major) all roll-forward to the next available SDK, 2.1.700, which is different to the previous experiments. Note that they don't use the highest feature version available, 2.1.801, they only roll forward to the next feature version, 2.1.700.

You could take these experiments further, but I think they demonstrate the patterns pretty well. That leaves just one final question…

Which roll-forward policy should you use?

In general, I suggest you don't use a global.json if you can help it. This effectively gives you the latestMajor policy by default, which uses the latest version of the .NET Core SDK, ensuring you get any associated bug fixes and performance improvements.

If you have to use a global.json and specify an SDK version for some reason, then I suggest you specify the lowest SDK version that works, and apply the latestMinor or latestFeature policy as appropriate. That will ensure your project can be built by the widest number of people (while still allowing you to control the range of SDK versions that are compatible).

Note that the flow charts and matching rules I've described above are specific to the .NET Core 3.x SDK. However, the matching rules for the highest SDK installed on your machine are used, so as long as you have any .NET Core 3.x SDK installed, they will apply to you.

Summary

In this post I looked in some depth at the new allowPrerelease and rollForward fields added to the global.json file in .NET Core 3.0. I described the algorithm used to determine which version, allowPrerelease, and rollForward values would be used, based on the presence of a global.json and whether or not you were running from Visual Studio. I then showed how each of the roll-forward policies affects the final selected SDK version.

Having the additional flexibility to define ranges of SDK versions is definitely useful, but should be used sparingly where possible. It can be easy to add accidental onerous requirements on people trying to build your project. Only add a global.json where it is necessary, and try to use permissive roll-forward policies like latestMajor or latestMinor where possible.