Generating argument expressions for minimal APIs (continued): Behind the scenes of minimal APIs - Part 5

In the previous post in this series, I showed how RequestDelegateFactory.CreateArgument() generates Expression trees for minimal API handlers. In this post I continue where we left off and look at more examples of the (effective) code that CreateArgument() generates to call your handlers.

I'm going to jump straight into it in this post, without any introduction. I strongly suggest reading the previous post if you haven't already (and obviously the earlier posts in the series if possible 😄)!

• Binding optional arrays of string, or StringValues? to the query string
• Binding required arrays of string or StringValues
• Binding int[] to the querystring
• Binding required services from DI
• Binding optional services from DI
• Binding to optional form files with IFormFileCollection and IFormFile
• Binding to a required form file with IFormFileCollection and IFormFile
• Binding BindAsync types
• Binding the request body to a type

Binding optional arrays of string, or StringValues? to the query string

We left off the last post having looked at binding both string and TryParse types like int parameters to the querystring. This was significantly simpler for string types where we have no conversion/parsing requirements.

The same goes when binding array parameters: binding string[] or StringValues is significantly simpler than binding arrays of TryParse types like int[].

StringValues is a readonly struct that represents zero/null, one, or many strings in an efficient way. It can be cast to and from both string and string[].

For example, consider the following endpoint, which would bind the q array to querystring values. For example, for a request like /?q=Mark&q=John the array would have two values, "Mark" and "John":

app.MapGet("/", (string[]? q) => {});

Essentially just like binding a string? in the previous post, the binding Expression can just grab the values from the querystring/route values/headers and pass them to the handler directly. The Query and Headers properties etc are directly convertible to a string[], so there's not much work to do:

Task Invoke(HttpContext httpContext)
{
    handler.Invoke(
        httpContext.Request.Query["q"] != null 
            ? httpContext.Request.Query["q"]
            : (string[]) null); 

    return Task.CompletedTask;
}

This generated code is essentially identical to when you're binding a string parameter; it can be used in both cases because the StringValues value returned from Query["q"] is directly convertible to both string and string[]!

Binding required arrays of string or StringValues

The next logical step is to make the parameter required instead of optional:

app.MapGet("/", (string[] q) => {});

In this case, the binding Expression is still simple in theory, but now the RequestDelegate needs to check that you definitely have a value in the querystring that matches. If not, the endpoint should return a 400 BadRequest response.

Task Invoke(HttpContext httpContext)
{
    bool wasParamCheckFailure = false; // Added by RequestDelegateFactory.Create()
    string[] q_local;

    q_local = httpContext.Request.Query["q"]
    if (q_local == null) // if the value is not provided
    {
        wasParamCheckFailure = true;
        Log.RequiredParameterNotProvided(httpContext, "string[]", "q", "query");
    }

    if(wasParamCheckFailure) // Added by RequestDelegateFactory.Create()
    {
        httpContext.Response.StatusCode = 400;
        return Task.CompletedTask;
    }

    handler.Invoke(q_local); 

    return Task.CompletedTask;
}

As an aside, there is an inconsistency in the behaviour of string[] and string[]? compared to other required and nullable parameter types. Theoretically, if you don't provide a value for the querystring, you might expect a string[]? parameter to have the value null (as suggested in the previous section), and a string[]? parameter to return a 400 Bad Request, as shown in the above code.

However, that's not the case. If you call these APIs and don't provide the expected querystring, the parameter will never be null and won't return a 400 Bad Request—instead you'll get an empty array. This currently goes against the documentation (and the behaviour for StringValues which does return the 400) so I've opened an issue about it here, which describes the cause of the difference. The consensus on that issue seems to be "by design, won't fix".

This is all looking pretty similar to the code you saw in the previous post for binding to string, which isn't that surprising given StringValues can be cast automatically to both string and string[]. Where things get trickier is if we have an array of types where you need to call TryParse.

Binding int[] to the querystring

In the next example, we're going to look at binding an int?[] parameter to the querystring:

app.MapGet("/", (int?[] q) => {});

In this case we need to first extract the string[] from the querystring into a temporary variable (tempStringArray), check whether each value is null, and if not, TryParse each of the values into an int. That makes the code quite a lot more complex!

Task Invoke(HttpContext httpContext)
{
    string tempSourceString; // Added by RequestDelegateFactory.Create()
    bool wasParamCheckFailure = false; // Added by RequestDelegateFactory.Create()
    int?[] q_local;
    
    // Parse the query into a temporary string[]
    string[] tempStringArray = httpContext.Request.Query["q"];

    if (tempStringArray != null)
    {
        // Initialize the variable that will hold the final result
        q_local = new int[tempStringArray.Length];

        int index = 0;
        while(true)
        {
            if(index < tempStringArray.Length)
            {
                // Loop through each string in the array
                tempSourceString = tempStringArray[index];

                // if the string is empty, e.g. in the URL /q=&q=
                // then leave the array entry as the default
                if(!string.IsNullOrEmpty(tempSourceString))
                {
                    // Try parsing the value
                    if (int.TryParse(tempSourceString, out var parsedValue))
                    {
                        // Parsed successfully, cast to int? and
                        q_local[i] = (int?)parsedValue;
                    }
                    else
                    {
                        // failed parsing
                        wasParamCheckFailure = true;
                        Log.ParameterBindingFailed(httpContext, "Nullable<int>[]", "query", tempSourceString, true);
                    }
                }
            }
            else
            {
                break;
            }
    
            index++
        }
    }
    else
    {
        // 👇 AFAICT, this code can't actually be hit due to 
        // https://github.com/dotnet/aspnetcore/issues/45956
        wasParamCheckFailure = true;
        Log.ParameterBindingFailed(httpContext, "Int32[]", "q", "query", tempSourceString, true);
    }

    if(wasParamCheckFailure) // Added by RequestDelegateFactory.Create()
    {
        httpContext.Response.StatusCode = 400;
        return Task.CompletedTask;
    }

    handler.Invoke(q_local); 

    return Task.CompletedTask;
}

You might well be thinking that this code looks a bit weird: a while loop with a nested if—why not just use a for or a foreach loop. My simplest guess: this code is easier to write when you're doing it all as Expressions! There's also a lot of duplicated Expression blocks between the various different parameter types, so I'm not inclined to criticise it. If you want to really understand, check out the 250 lines of Expression gymnastics in BindParameterFromValue()!

Binding required services from DI

After that complexity, lets look at some simpler ones: binding to services in the DI container. For example, take the following example program:

var builder = WebApplication.CreateBuilder();
builder.Services.AddSingleton<MyService>();

var app = builder.Build();
app.MapGet("/", (MyService service) => {});
app.Run()

class MyService {}

The minimal API endpoint automatically detects that the MyService type is available in DI, and binds the handler parameter to the service using GetRequiredService<T>()

async Task Invoke(HttpContext httpContext)
{
    handler.Invoke(
        httpContext.RequestServices.GetRequiredService<MyService>()); 

    return Task.CompletedTask;
}

Binding optional services from DI

"Optional" services, where the service may or may not be registered in DI generally feel like a bad pattern to me, but they're supported directly by minimal APIs. You'll typically need to use the [FromServices] attribute so that minimal APIs knows it's a DI service and mark the parameter as optional with ?:

var builder = WebApplication.CreateBuilder();
var app = builder.Build();
app.MapGet("/", ([FromServices] MyService? service) => {});
app.Run()

The resulting Expression is very similar to the "required" case. The only difference is optional services use GetService<T> instead of GetRequiredService<T>.

async Task Invoke(HttpContext httpContext)
{
    handler.Invoke(
        httpContext.RequestServices.GetService<IService>()); 

    return Task.CompletedTask;
}

Binding to optional form files with IFormFileCollection and IFormFile

Next up we'll look at binding IFormFile and IFormFileCollection. You may remember from a previous post that these are two of the "well-known" types that minimal APIs can bind to directly. They're roughly the same in terms of how they bind: IFormFileCollection binds to all the form files, while IFormFile binds to a single file, named "file" in the following example:

app.MapGet("/", (IFormFile? file) => {});

Form files are exposed directly in ASP.NET Core on the HttpRequest.Form.Files property, so the Expression is very similar to when we were binding a string or StringValues property to the querystring:

Task Invoke(HttpContext httpContext)
{
    handler.Invoke(
        httpContext.Request.Form.Files["file"] != null 
            ? httpContext.Request.Form.Files["file"]
            : (IFormFile) null); 

    return Task.CompletedTask;
}

What isn't shown in the above delegate is the code that actually reads the Form from the body as the first step of the final RequestDelegate. The process for doing this is a bit convoluted, but it essentially calls the following TryReadFormAsync method, and if that doesn't indicate success, then immediately returns.

static async Task<(object? FormValue, bool Successful)> TryReadFormAsync(
    HttpContext httpContext,
    string parameterTypeName,
    string parameterName,
    bool throwOnBadRequest)
{
    object? formValue = null;
    var feature = httpContext.Features.Get<IHttpRequestBodyDetectionFeature>();

    if (feature?.CanHaveBody == true)
    {
        if (!httpContext.Request.HasFormContentType)
        {
            Log.UnexpectedNonFormContentType(httpContext, httpContext.Request.ContentType, throwOnBadRequest);
            httpContext.Response.StatusCode = StatusCodes.Status415UnsupportedMediaType;
            return (null, false);
        }

        try
        {
            formValue = await httpContext.Request.ReadFormAsync();
        }
        catch (IOException ex)
        {
            Log.RequestBodyIOException(httpContext, ex);
            return (null, false);
        }
        catch (InvalidDataException ex)
        {
            Log.InvalidFormRequestBody(httpContext, parameterTypeName, parameterName, ex, throwOnBadRequest);
            httpContext.Response.StatusCode = StatusCodes.Status400BadRequest;
            return (null, false);
        }
    }

    return (formValue, true);
}

I haven't shown how this method is invoked as things get a bit complicated with Expression compilation and nested delegates, but you'll see a similar approach when we come to binding the body.

Binding to a required form file with IFormFileCollection and IFormFile

For completeness, we'll also look at binding to a required IFormFile or IFormFileCollection, where we need to check that the file(s) is actually sent. I've used IFormFileCollection in this example for variety.

app.MapGet("/", (IFormFileCollection files) => {});

The result is a pattern that you've seen many times now: we bind the property and check it for null. If it's null, we return a 400, if it's not, we pass it as the argument to the handler.

Task Invoke(HttpContext httpContext)
{
    bool wasParamCheckFailure = false; // Added by RequestDelegateFactory.Create()
    IFormFileCollection files;

    files = httpContext.Request.Form.Files
    if (files == null)
    {
        wasParamCheckFailure = true;
        Log.RequiredParameterNotProvided(httpContext, "IFormFileCollection", "files", "body");
    }

    if(wasParamCheckFailure) // Added by RequestDelegateFactory.Create()
    {
        httpContext.Response.StatusCode = 400;
        return Task.CompletedTask;
    }

    handler.Invoke(files); 

    return Task.CompletedTask;
}

TryReadFormAsync() is called at the start of the RequestDelegate, as shown in the previous example.

Ok, that's it, we're out of "easy" binding expressions now. Back to the harder stuff…

Binding BindAsync types

Up to this point, I've been showing examples of the "final" RequestDelegate generated by RequestDelegateFactory.Create(), using the parameter Expression definitions created in the CreateArgument() method. But I've been lying somewhat; the RequestDelegate often isn't as "clean" as the examples I've shown, for technical reasons.

That doesn't matter for the most part, which is why I opted for this approach. But for the rest of this post I'm going to hew a little closer to reality.

We'll start with BindAsync types. As you may remember from a previous post, types can control their binding completely by implementing one of the following static methods:

public static ValueTask<T?> BindAsync(HttpContext context, ParameterInfo parameter);
public static ValueTask<T?> BindAsync(HttpContext context);

Lets imagine we have a type MyType, that has a BindAsync method that we use in a minimal API:

app.MapGet("/", (MyType t) => {});

To generate the parameter expression for the parameter t, CreateArgument() calls the method BindParameterFromBindAsync(). This creates a Func<HttpContext, ValueTask<object?>> that invokes the BindAsync method of our type, something a bit like this:

// This is very simplified, for more details see 
// https://github.com/dotnet/aspnetcore/blob/v7.0.1/src/Shared/ParameterBindingMethodCache.cs#L195
MethodInfo bindAsyncMethod = typeof(MyType).GetMethod("BindAsync");

var bindAsyncDelegate = Expression.Lambda<Func<HttpContext, ValueTask<object?>>>(
    bindAsyncMethod, HttpContextExpr).Compile();

The end result is a Func<> which, when called, creates an instance of MyType given an HttpContext parameter (or which may throw an Exception).

The CreateArgument() method adds the bindAsyncDelegate to the RequestDelegateFactoryContext.ParameterBinders collection. RequestDelegateFactory.Create() then uses this list to bind all of the BindAsync-able parameters before calling the "body" of the RequestDelegate.

Most of the following RequestDelegate is created in the Create() method rather than CreateArgument(), but I've included the whole thing so you can see how it all fits together. I've commented the code of course, but there are a few interesting points here:

• There's a slightly confusing mix of code created using Expression and compiling, and simple Func<> closures. The code I show below is a best attempt to show the result after compiling the Expression and combining with the various Func<>, but it's not a clear one-to-one with the real code.
• The inner Func<> (that I've called generateResult uses captured closure values. I've glossed over this in previous cases, but it's clear that the ParameterBindings collection is captured directly in this case.
• The target parameter in the inner continuation is the instance on which the handler method is defined. For lambda endpoint handlers, this is the generated closure class for the delegate, but it may also be null for static handlers.

As most of this is created by RequestDelegateFactory.Create(), I've highlighted the parts that are created by CreateArgument specifically.

// These are created outside the RequestDelegate method and are captured 
// by the RequeustDelegate Func<> closure
var binders = factoryContext.ParameterBinders.ToArray();
// The length is equal to the number of parameters that implement BindAsync
var count = binders.Length;

async Task Invoke(HttpContext httpContext)
{
    // the target object passed here is the "handler" object, if there is one
    var Task Invoke(object? target, HttpContext httpContext)
    {
        // This array holds the array of parameter values
        // that are bound using BindAsync to the HttpContext
        var boundValues = new object?[count];
        for (var i = 0; i < count; i++)
        {
            // Invoke the Func<> for each parameter, and record the result
            // For this example, it creates a single MyType object
            boundValues[i] = await binders[i](httpContext);
        }

        // define the continuation that's invoked after the binding
        var generateResult = (object? target, HttpContext httpContext, object?[] boundValues) =>
        {
            bool wasParamCheckFailure = false;

            // This check is only emitted if the parameter is required
            // and is one of the few parts emitted by CreateArgument()
            if(boundValues[0] == null)
            {
                wasParamCheckFailure = true;
                Log.RequiredParameterNotProvided(httpContext, "MyType", "t", "MyType.BindAsync(httpContext)", true);
            }

            if(wasParamCheckFailure)
            {
                httpContext.Response.StatusCode = 400;
                return Task.CompletedTask;
            }

            // CreateArgument() emits the expression that casts 
            // the bound value to the parameter type, MyType
            target.handler.Invoke((MyType)boundValues[0]); 

            return Task.CompletedTask;
        };

        // invoke the Func<> passing the values from the outer function
        generateResult(target, httpContext, boundValues);
    }

    // Invoke the inner method, passing in the lambda closure instance as the target
    return Invoke(target: Program.<>c, httpContext);
}

If you're confused, I don't blame you. There's a lot of seemingly unnecessary "wrapping" in inner Func variables in the example above, which makes things a bit harder to follow. In previous examples I "flattened" these Func<> calls, but I left them in here for completeness. I'm still undecided whether I regret that choice 😅

Note that the real RequestDelegate generated by RequestDelegateFactory doesn't have the Func<> variables above like generateResult etc. Expressions are compiled to Func<>, and invoked directly when building up the sections. The above is my attempt to represent that without getting hung up on the detail too much!

Binding the request body to a type

In the above example I let all the Func<> ugliness show; for the next binding I'll show some of the ugliness where necessary, but gloss over the rest. You'll see what I mean…

Let's imagine we have the same API, but this time the MyType type doesn't implement BindAsync:

app.MapGet("/", (MyType t) => {});

By default, this API binds the t parameter to the request's body. This is the first thing that occurs in the generated RequestDelegate, which calls the helper function TryReadBodyAsync. As before, almost all of this is created by RequestDelegateFactory.Create() rather than CreateArgument(), but I've included it here for completeness.

async Task Invoke(HttpContext httpContext)
{
    // try to read the body
    var (bodyValue, successful) = await TryReadBodyAsync(
        httpContext,
        typeof(MyType),
        "MyType",
        "t",
        false, // factoryContext.AllowEmptyRequestBody
        true); // factoryContext.ThrowOnBadRequest

    // if the read was not successful, bail out
    if (!successful)
    {
        return;
    }

    // define the Func<> that calls the handler
    var generateResponse = (object? target, HttpContext httpContext, object? bodyValue) =>
    {
        bool wasParamCheckFailure = false;
        
        if (bodyValue == null) // this condition is created in CreateArgument()
        {
            wasParamCheckFailure = true;
            Log.ImplicitBodyNotProvided(httpContext, "MyType", true);
        }

        handler.Invoke((MyType)bodyValue);  // cast created in CreateArgument()

        return Task.CompletedTask;
    };

    // Invoke the continuation, passing null as the target
    await generateResponse(target: null, httpContext, bodyValue);
}

// This helper function reads the request body and creates 
// an instance of the object using System.Text.Json
static async Task<(object? FormValue, bool Successful)> TryReadBodyAsync(
    HttpContext httpContext,
    Type bodyType,
    string parameterTypeName,
    string parameterName,
    bool allowEmptyRequestBody,
    bool throwOnBadRequest)
{
    object? defaultBodyValue = null;

    if (allowEmptyRequestBody && bodyType.IsValueType)
    {
        defaultBodyValue = CreateValueType(bodyType);
    }

    var bodyValue = defaultBodyValue;
    var feature = httpContext.Features.Get<IHttpRequestBodyDetectionFeature>();

    if (feature?.CanHaveBody == true)
    {
        if (!httpContext.Request.HasJsonContentType())
        {
            Log.UnexpectedJsonContentType(httpContext, httpContext.Request.ContentType, throwOnBadRequest);
            httpContext.Response.StatusCode = StatusCodes.Status415UnsupportedMediaType;
            return (null, false);
        }
        try
        {
            bodyValue = await httpContext.Request.ReadFromJsonAsync(bodyType);
        }
        catch (IOException ex)
        {
            Log.RequestBodyIOException(httpContext, ex);
            return (null, false);
        }
        catch (JsonException ex)
        {
            Log.InvalidJsonRequestBody(httpContext, parameterTypeName, parameterName, ex, throwOnBadRequest);
            httpContext.Response.StatusCode = StatusCodes.Status400BadRequest;
            return (null, false);
        }
    }

    return (bodyValue, true);
}

One thing not shown in the code above is the fact that CreateArgument() adds additional [Accepts] metadata to the metadata collection as part of InferMetadata call, indicating that the endpoint expects a JSON MyType request body.

This pattern of nested Func<> calls is how the TryReadFormAsync() method is called when binding IFormFile. The pattern for reading the body is identical; it's only the processing of the body that differs.

Phew. We're done. There are extra permutations we could look at: for each parameter type we could compare required vs. optional vs. default values, but we've already looked at representative examples of this, so there's not a huge amount to be gained.

The main takeaways from the post are that CreateArgument() sometimes does a lot of work. The Expression to read an int[] parameter, for example, is substantial. In contrast, binding to a DI service is a trivial Expression!

In the next post, we're going to take a look at another part of the RequestDelegate expression generation: the methods that control how to handle the return value of the handler endpoint. So far I've limited the examples to returning void, but you can return anything from your APIs.

Summary

The RequestDelegateFactory.CreateArgument() method is responsible for creating the Expression trees for binding minimal API handler arguments to the HttpContext. RequestDelegateFactory.Create() uses these expression trees to build the final RequestDelegate that ASP.NET Core executes to handle a request.

In the previous post and in this post I showed examples of some of the expression trees generated for specific parameter types. In this post I started by showing binding arrays to the querystring, and then showed binding DI services and IFormFile files.

Next we looked at the more complex examples of binding parameters using BindAsync, and parameters that bind to the request body. The resulting RequestDelegate generated in both of these cases requires a lot of code generated outside of CreateArgument(), but I included it for completeness.

The example code shows how several nested Func<> are generated and invoked. This is closer to the "real" behaviour of the compiled RequestDelegate, but it does make for more confusing reading!

In the next post we'll look at another section of the generated RequestDelegate: handling the return type of a minimal API and writing the response.