Using Unix domain sockets with ASP.NET Core and HttpClient

In this post I describe Unix domain sockets, what they are, the scenarios where they're useful, how to use them with ASP.NET Core, and how to call a UDS ASP.NET Core app using HttpClient. For those who read my previous post, this post is going to be very familiar!

What are Unix Domain Sockets?

A Unix domain socket (UDS) is an endpoint for communication between processes on the same host. It's very similar to an IP socket that you'd use for normal internet traffic, but communication happens entirely on one host, inside the operating system's kernel. As the name suggests, UDS is a standard part of POSIX operating systems like Linux and macOS, but interestingly UDS is also supported in Windows from Windows 10 and Windows Server 2019!

There are several types of UDS, each of which is comparable to a protocol in the TCP/IP suite:

• SocketType.Stream (SOCK_STREAM)—comparable to TCP; a stream-oriented socket.
• SocketType.Dgram (SOCK_DGRAM)— comparable to UDP; datagram-oriented, connectionless messages, which are theoretically unreliable, though in most implementations they are reliable.
• SocketType.Seqpacket (SOCK_SEQPACKET)—comparable to SCTP; connection-based, reliable, and ordered messages.

On Windows, only SocketType.Stream is available; Dgram and Seqpacket are not supported.

The UDS implementation on Linux supports sending file descriptors and credentials over UDS, as away of providing access to resources that the receiving process would not normally have. This isn't supported on Windows.

In my previous post, I described that you specify a Windows named pipes by giving it a unique name, using the format \\<ServerName>\pipe\<PipeName>. Unix domain sockets are similarly defined by a path; the UDS path is literally a file system path. For example, you might use

• /var/run/myapp/test.socket on Linux/macOS
• C:\Users\Me\AppData\Local\Temp\MyApp\test.socket on Windows

Note that the path uses the standard operating system conventions for defining file paths. The path is also subject to standard user permissions, so you can only create a socket at a path where you would also be able to create a file.

The .socket suffix is not required but using that or .sock seems to be somewhat of a convention.

There's not much more to think about with UDS, so in the next section we'll look at why you might want to use them.

Why use Unix domain sockets?

Unix domain sockets allow for inter-process communication on a single machine. So why would you choose them over TCP/IP, for example, where you could use the loopback address (localhost) for single-server communication? And on Windows, why would you choose them over Windows named pipes?

In general, there are several reasons you might choose to use UDS instead of TCP/IP for inter-process communication:

• Unix domain sockets generally have less overhead and faster transfer speeds than using TCP/IP
• TCP/IP sockets are a limited resource, whereas there's no hard limit for Unix domain sockets
• Unix domain sockets appear as files, so it's easy to do "discovery" of well-known paths
• Integration with the file system also adds an additional security layer (if you can't access the file path, you can't access the socket)

The first point is an easy win—a quick googling shows that benchmarks for UDS vs TCP/IP always fall down on the side of UDS, giving significantly lower latency as well as significantly higher throughput. This stems primarily from the fact that UDS is optimised for same-server communication, whereas IP communication over localhost must navigate the IP stack on both the send and receive side.

TCP/IP sockets are a limited resource; you can only have a maximum of 65,535 sockets in use at any time. When you add in the TIME_WAIT issue, the actual maximum number of active connections could be considerably smaller than this. A localhost connection similarly consumes from this pool. Using UDS neatly sidesteps this issue; it allows communication without using up TCP/IP sockets.

One neat use case for UDS that we use in our work at Datadog is automated discovery. The Datadog trace-agent listens for UDS requests at a well-known file path, /var/run/datadog/apm.socket. The .NET client library can do a simple File.Exists() check of this path during initialization, and can immediately know whether the endpoint is available, or whether we should fallback to TCP/IP, for example.

Obviously you can send a test request to a localhost address too, it's just going to be slower than a quick File.Exists()

Finally, Unix domain socket file paths can be protected using standard file permissions, so that only processes with access to a file at the UDS path can access the socket. One other neat part of the file-system address space is that you can do things like share sockets between docker containers by sharing a volume mount—very cool!

So when should you choose Unix domain sockets over Windows named pipes? Both can be used for inter-process communication on a single machine, and they both have many of the same advantages compared to using localhost TCP/IP requests. Which you choose will depend on what features you require. UDS is the only cross platform option, which may be appealing if your app is cross-platform. But if you're using and Windows and need "UDP" (Datagram) support rather than just TCP (Stream), then you will need to use Windows named pipes as UDS on Windows doesn't support it.

Creating a UDS server with ASP.NET Core and Kestrel

Unlike Windows named pipes, which were only added to ASP.NET Core in .NET 8, ASP.NET Core has supported UDS since ASP.NET Core 2.0! Changing your application to listen using UDS instead of TCP is very similar to the approaches I showed in my previous post on windows named pipes. You must do one of the following:

• Configure Kestrel in code using ListenUnixSocket()
• Set the URLs for your application to http://unix:<path-to-socket>

The example below shows a very simple .NET 8 test app (adapted from the empty template: dotnet new web). The only change here is that I specifically configured the app to listen using UDS:

var builder = WebApplication.CreateBuilder(args);

// Create a path to use for the UDS
// Something like: C:\Users\Sock\AppData\Local\Temp\test.socket
var socketPath = Path.Combine(Path.GetTempPath(), "test.socket");

// 👇 Configure Kestrel to listen at the UDS path
builder.WebHost.ConfigureKestrel(
    opts => opts.ListenUnixSocket(socketPath));

var app = builder.Build();

app.MapGet("/test", () =>  "Hello world!");

app.Run();

If you start the app with dotnet run, you'll see that the logs show the listen address as:

info: Microsoft.Hosting.Lifetime[14]    
      Now listening on: http://unix:C:\Users\Sock\AppData\Local\Temp\test.socket

The http://unix: looks a little odd, but it kind of makes sense. After all we're still sending HTTP requests, we're just sending them over a Unix domain socket instead of a TCP socket!

As I described in my previous posts, you can also configure Kestrel entirely using IConfiguration, so instead of adding the ConfigureKestrel() call above, you could instead define the listen URL in your appsettings.json file like this:

{
  "Kestrel": {
    "Endpoints": {
      "NamedPipeEndpoint": {
        "Url": "http://unix:C:\\Users\\Sock\\AppData\\Local\\Temp\\test.socket"
      }
    }
  }
}

Take care with escaping the slashes if you're using UDS on Windows as I am in the example above!

You can also use all the other approaches I described in my previous post to opt-in to binding UDS using the same binding address pattern as the above configuration does. For example, in the linux example below:

export ASPNETCORE_URLS="http://unix:/var/opt/test.socket"

And in case you were wondering, yes, you can also specify https in your URLs and kestrel will use HTTPS for the requests!

Calling the UDS server with HttpClient

Creating an HTTP server (or gRPC server, or anything else supported by ASP.NET Core!) that listens over UDS is surprisingly easy. But you will likely also want to be able to send requests to the server, so we need a client.

For this, we need our trusty HttpClient and SocketsHttpHandler, combined with UnixDomainSocketEndPoint. UnixDomainSocketEndPoint and SocketsHttpHandler were introduced in .NET Core 2.1, but we need to use the SocketsHttpHandler.ConnectCallback property specifically, which was only added in .NET 5.

To configure an HttpClient to use UDS you need to specify a custom ConnectCallback() which will create a UnixDomainSocketEndPoint instance and connect to the server. After configuring the client, you just use it like you would calling any ASP.NET Core app!

using System.IO.Pipes;

var httpHandler = new SocketsHttpHandler
{
    // Called to open a new connection
    ConnectCallback = async (ctx, ct) =>
    {
        var socketPath = @"C:\Users\Sock\AppData\Local\Temp\test.socket";
        // Define the type of socket we want, i.e. a UDS stream-oriented socket
        var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);

        // Create a UDS endpoint using the socket path
        var endpoint = new UnixDomainSocketEndPoint(socketPath);
        
        // Connect to the server!
        await socket.ConnectAsync(endpoint, ct);
        
        // Wrap the socket in a NetworkStream and return it
        // Setting ownsSocket: true means the NetworkStream will 
        // close and dispose the Socket when the stream is disposed
        return new NetworkStream(socket, ownsSocket: true);
    }
};

// Create an HttpClient using the UDS handler
var httpClient = new HttpClient(httpHandler)
{
    BaseAddress = new Uri("http://localhost"); // 👈 Use localhost as the base address
};

var result = await httpClient.GetStringAsync("/test");
Console.WriteLine(result);

The code for connecting to the socket isn't necessarily obvious unless you're used to working directly with Sockets, but otherwise the HttpClient code is relatively standard. The main "oddity" is that you need to give use a valid hostname in the BaseAddress of the HttpClient, or alternatively in the GetStringAsync() if you don't specify a BaseAddress. I've used localhost for simplicity, but it doesn't really matter what we use here. If you run the app, it should print "Hello World!" 🎉

Remember to use https: in the BaseAddress if you have configured ASP.NET Core to listen for HTTPS over UDS!

And that's all there is to it. Using UDS with ASP.NET Core isn't much harder than using TCP/IP, and if you only need to send data between processes on the same host then UDS may be a faster and more reliable option. What's more, UDS is well tested, and available on Linux, macOS and Windows, despite the name!😀

Summary

In this post I described Unix domain sockets (UDS). I discussed what they are, how they work, and some of the scenarios where they may be useful. They're particularly useful when you're running in scenarios in which cross-process TCP communication is problematic. Next I described the UDS support in ASP.NET Core and how to configure your app to listen to a UDS path. Finally, I showed how you can use HttpClient with a custom SocketsHttpHandler.ConnectCallback and UnixDomainSocketEndPoint to make requests to a UDS HTTP server.