Description
Background and motivation
Similarly to HTTP/2, we want to introduce support for multiple HTTP/3 connections (#101535). In order to properly manage the available streams count, we need the following API additions.
QUIC protocol has baked in stream multiplexing into the transport, meaning it handles stream creation and enforcement of stream limits. This is a difference from HTTP/2 where it's the application protocol that handles that.
QUIC stream limits are exposed as an ever increasing stream id and not as a stream count. For example, QUIC server will send that it accepts streams up to id 20. The client cannot open additional streams even after closing all of the streams, it has to wait until the server sends a new, incremented max stream id (see QUIC MAX_STREAMS frame). Which leads to the shape of the API change based on the protocol definition: callback to listen to newly released stream limit so that we can use the connection for new requests.
Original request for multiple HTTP/3 connections: #51775.
Draft PR with real usage: #101531.
API Proposal
namespace System.Net.Quic;
// Existing class.
public abstract partial class QuicConnectionOptions
{
// New callback, raised when MAX_STREAMS frame arrives, reports new capacity for stream limits.
// First invocation with the initial values might happen during QuicConnection.ConnectAsync or QuicListener.AcceptConnectionAsync, before the QuicConnection object is handed out.
public QuicConnectionStreamCapacityAvailableCallback? StreamCapacityAvailableCallback { get; set; }
}
// Callback definition:
// The arguments represent additional counts of released stream limits.
// The initial values will be reported by the first invocation of the callback.
public delegate void QuicConnectionStreamCapacityAvailableCallback(QuicConnection connection, QuicConnectionStreamCapacityAvailableArgs args);
// Callback arguments:
public readonly struct QuicConnectionStreamCapacityAvailableArgs
{
public readonly int BidirectionalStreamsCountIncrement { get; init; }
public readonly int UnidirectionalStreamsCountIncrement { get; init; }
}
API Usage
Simplified logic from Http3Connection
:
// Method that creates Http3Connection.
// Shows the slight disadvantage of passing callback in the options class together with not having public QuicConnection ctor.
// As a result, H/3 connection cannot take QuicConnection in a constructor argument and must be instantiated upfront, and only after ConnectAsync finishes, tied with QuicConnection.
Http3Connection h3Connection = new Http3Connection();
// The first invocation of h3Connection.StreamCapacityAvailableCallback will likely happen during QuicConnection.ConnectAsync before it returns the QuicConnection instance.
QuicConnection quicConnection = await QuicConnection.ConnectAsync(new QuicClientConnectionOptions()
{
// Other options ...
StreamCapacityAvailableCallback = h3Connection.StreamCapacityAvailableCallback
}, cancellationToken).ConfigureAwait(false);
h3Connection.InitQuicConnection(quicConnection);
internal class Http3Connection
{
private int _availableRequestStreamsCount;
// Using the available stream count to see whether we can process the request on the current HTTP/3 connection.
public bool TryReserveStream()
{
lock (SyncObj)
{
if (_availableRequestStreamsCount == 0)
{
return false;
}
--_availableRequestStreamsCount;
return true;
}
}
// In case the TryReserveStream returns false, we remove the HTTP/3 connection from the pool of usable connections and wait for StreamCapacityAvailableCallback event to put it back.
public Task WaitForAvailableStreamsAsync()
{
lock (SyncObj)
{
if (_availableRequestStreamsCount > 0)
{
return Task.FromResult(true);
}
_availableStreamsWaiter = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
return _availableStreamsWaiter.Task;
}
}
// Callback from the QUIC will increment the available stream count and unblock a waiter if there's one.
public void StreamCapacityAvailableCallback(QuicConnection sender, QuicConnectionStreamCapacityAvailableArgs args)
{
lock (SyncObj)
{
_availableRequestStreamsCount += args.BidirectionalStreamsCountIncrement;
_availableStreamsWaiter?.SetResult();
_availableStreamsWaiter = null;
}
}
}
Alternative Designs
Using cumulative counts (not increments) in the available(uni|bi)directionalStreamsCount
values
We could use the cumulative value for availableBidirectionalStreamsCount
/ availableUnidirectionalStreamsCount
(corresponding to MAX_STREAMS). And the user would keep the cumulative value of opened + reserved streams. Also the user would need to make sure to ignore any lower value as the callbacks might swap order.
Callback arguments as individual parameters:
public delegate void QuicConnectionStreamCapacityAvailableCallback(QuicConnection connection, int bidirectionalStreamsCountIncrement, int unidirectionalStreamsCountIncrement);
Most of our callback do not wrap args into structs, it's easier to consume, but it isn't in any way extendable.
Using Event / Callback property on QuicConnection
The events are not very widespread tool used by .NET libraries. There are few in networking, but not much and nothing recent has been designed with them.
// Existing class.
public abstract partial class QuicConnectionOptions
{
// New event, raised when MAX_STREAMS frame arrives, reports new capacity for stream limits.
public event System.Net.Quic.QuicConnectionStreamCapacityAvailableEventHandler? StreamCapacityAvailable { add { } remove { } }
}
// Arguments for StreamCapacityAvailable event.
public partial class QuicConnectionStreamCapacityAvailableEventArgs : System.EventArgs
{
internal QuicConnectionStreamCapacityAvailableEventArgs() { }
// The additional count of released stream limit. Corresponds to new_value(AvailableBidirectionalStreamsCount) - old_value(AvailableBidirectionalStreamsCount).
public int BidirectionalStreamsCountIncrement { get { throw null; } }
// The additional count of released stream limit. Corresponds to new_value(AvailableUnidirectionalStreamsCount) - old_value(AvailableUnidirectionalStreamsCount).
public int UnidirectionalStreamsCountIncrement { get { throw null; } }
}
// Event delegate definition.
public delegate void QuicConnectionStreamCapacityAvailableEventHandler(object? sender, System.Net.Quic.QuicConnectionStreamCapacityAvailableEventArgs e);
Using Tasks (alternative to events / callbacks)
It's possible to replace StreamCapacityAvailable
with Tasks, one for each stream type:
public sealed partial class QuicConnection : System.IAsyncDisposable
{
public Task<int> BidirectionalStreamsCapacityAvailable { get; }
public Task<int> UnidirectionalStreamsCapacityAvailable { get; }
}
Disadvantage is that they have to be recreated and re-registered after each completion.
Risks
This is an advanced API that's being tailored mainly for H/3 multiple connection support (we will document this as such). The potential users must take care when designing logic around this and correctly keep the number of opened streams (decrementing the capacity).