HttpClient buffer size limit exceeded

Question

I am using my client to get info about a file stored in my Swift Object Storage which can be accessed through a REST API. In Swift the HEAD method and URL leading to the object returns its metadata (hash, timestamp, etc.) contained in the HTML response's (has no content) headers. My code works when file size is < 2GB. I get the HttpResponseMessage and am able to parse it for data, but with a file > 2GB I get:

Cannot write more bytes to the buffer than the configured maximum buffer size: 2147483647

The HttpClient.MaxResponseContentBufferSize property cannot be set to a value > 2GB but I don't want its content. Is there a way to solve this?

public HttpResponseMessage FileCheckResponse(string objectName) {
    //create url which will be appended to HttpClient (m_client)
    string requestUrl = RequestUrlBuilder(m_containerName, objectName);
    //create request message with Head method
    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Head, requestUrl);
    //exception thrown here...
    HttpResponseMessage response = m_client.SendAsync(request).Result;

    return response;
}

The same action using Dev HTTP Client (Chrome extension) has no problem. It seems the Content-Length header makes it unfeasible. Output from Dev HTTP Client:

Content-Length: 3900762112
Accept-Ranges: bytes
Last-Modified: Fri, 06 Sep 2013 16:24:30 GMT
Etag: da4392bdb5c90edf31c14d008570fb95
X-Timestamp: 1378484670.87557
Content-Type: application/octet-stream
Date: Tue, 10 Sep 2013 13:25:27 GMT
Connection: keep-alive

Do not include solution to question please (post a separate answer instead). — user4157124
– user4157124, Commented Oct 15 at 6:23

Darrel Miller · Accepted Answer · 2013-09-10 13:54:48Z

44

Use the SendAsync overload that allows you to specify the HttpCompletionOptions. With this you can tell HttpClient not to create a buffer for the response contents.

answered Sep 10, 2013 at 13:54

Darrel Miller

143k32 gold badges202 silver badges246 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

gi097 Over a year ago

This worked for me: await _client.GetAsync(uri, HttpCompletionOption.ResponseHeadersRead);

Vedran Over a year ago

Doesn't work for me, still getting the System.Net.Http.HttpRequestException: Cannot write more bytes to the buffer than the configured maximum buffer size

aronccs Over a year ago

How can I do that for PostAync?

Darrel Miller Over a year ago

@aronccs Create a HttpRequestMessage, set the method to POST and pass it to SendAsync with the appropriate HttpCompletionOption.

aronccs Over a year ago

@DarrelMiller what if I have data to send to the POST?

|

Per Johansson · Accepted Answer · 2020-11-24 11:32:24Z

1

Another way to increase buffer size is

HttpClient client = new HttpClient();
client.MaxResponseContentBufferSize = 9999999;

answered Nov 24, 2020 at 11:32

Per Johansson

1571 gold badge2 silver badges5 bronze badges

4 Comments

Alex S. Over a year ago

Am I correct it's still limited by Int32 MaxValue?

Tanmoy Sengupta Over a year ago

Alex S., MaxResponseContentBufferSize is a long data type which is Int64, so it would be limited by Int64 MaxValue.

Clay Fowler Over a year ago

It is limited to Int32.MaxValue (2GB) even though the type is long: github.com/mono/mono/issues/14214#issuecomment-497445929

Per Lundberg Oct 15 at 5:21

For reference, still seems to be limited to 2GB with .NET 9.

Collectives™ on Stack Overflow

HttpClient buffer size limit exceeded

2 Answers 2

6 Comments

4 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

4 Comments

Linked

Related