Should client always consume response body ?

May 28, 2014 at 8:06 AM
Question related to resources freeing.
A client issue a request to a server, and checks response (status, headers, etc.).
If status is bad, it throws away (destroy) the http_reponse object [this may also occur in case of exception during response validation].
So the response body is never read, and I wonder if it should be read before destroying response object. From my observations, this seems to be true (at least under Linux) otherwise some tasks seem to remain active.
However I can't see any way to do that, besides getting body stream and reading up to the end (that's painful). Is there a method to close() the body without reading it (ie discard connection in pool, close socket) ?

Mainly concerned by Windows 7 and 8 platforms BTW.
Coordinator
May 29, 2014 at 9:51 PM
Hi momaison,

Let me explain a little bit how the HTTP response flow works with our library. I realize there should be documented better, eventually we will get up a tutorial or document describing this is more detail.

The task returned from http_client::request(...) is completed once the HTTP headers have arrived, it doesn't wait for body of the response to arrive as well. This allows for an opportunity for scenarios where streaming or dealing with large amounts of data can be handled. The task returned from the http_response::content_ready() and http_response::extract_*() methods will wait until the entire response body has arrived before completing.

In regards to errors and exceptions if an error occurs before the HTTP headers have completely arrived then an exception will be thrown out of a task from the http_client::request(...) function. All applications need to handle to properly handle exceptions in this case. If an error occurs after the HTTP headers have arrived it will result in an exception being throw from http_response::content_ready, the http_response::extract_* functions, or the response stream body. Exceptions at this point do not need to be properly observed and handled. We didn't want to make it a requirement that all requests must consume and read all response bodies. Please note if an error occurs and you actually are trying read the response body then you will see an exception.

So if you want or need to completely wait for a request to be completed with the entire response body completed you will need to wait or hook up a continuation to the task returned from http_response::content_ready() or the http_response::extract_* functions. Also FYI if you want to cancel a request you could do so using a cancellation_token that can be passed to the http_client::request(...) methods.

Does that help explain?
Steve
Jun 2, 2014 at 3:33 PM
Hi Steve,

Thank you for these informations, however I'm afraid my question is still opened.
My concern was mainly regarding http_response object destruction.
Code sample:

http_client c( ... );
http_request req( ... );
http_response resp = c.request( req ).get();
// here response headers have been read.
// Is it always legal to destroy 'resp' object, without calling any of content_ready() or extract_* methods ?
// Is it legal only if the body is empty ?

"We didn't want to make it a requirement that all requests must consume and read all response bodies". That's nice idea, however under Linux some resources/threads seem to remain active at this step, until body is consumed (as reported by valgrind). Can't really know under Windows.

Regarding cancellation tokens, would it be an error to cancel an already finished request (ie response consumed) ? For example in case token is canceled in a destructor.
Coordinator
Jun 2, 2014 at 10:02 PM
Hi momaison,

Let me try and answer those more directly :).

"// Is it always legal to destroy 'resp' object, without calling any of content_ready() or extract_* methods ?"

The http_response object is actually internally a shared_ptr using the pointer to implementation pattern. Internally we keep a reference on the underlying http_response until it the response has entirely been read in to completion or been canceled. Yes it is legal to not have to call the methods content_ready or extract_* your program won't crash, however it you want to make sure all network activity and tasks are finished running then you must wait on the returned tasks from those methods or performance cancellation.

"// Is it legal only if the body is empty ?"

If there is no response body then the entire request should be processed by the time the task from http_client::request(...) is completed. So yes you don't have to call the content_ready() or extract_* functions.

"Regarding cancellation tokens, would it be an error to cancel an already finished request (ie response consumed) ?"

The nature of cancellation is that it is a race condition. It is entirely possible that by the time you call cancel the response body could have already been entirely read in. If you try to cancel the request once the entire response body has arrived then nothing will happen. If you cancel before the response body has completed then an exception will be thrown from the methods context_ready() and extract_* indicating cancellation.

In general if you won't wait or cancel then yes there could be tasks/IO being performed for the outstanding request. This won't cause any crashes or issues in your program unless you are about to do something like exit the process. In that case you should wait/cancel uncompleted requests/responses.

Steve
Jun 3, 2014 at 7:36 AM
Thank you for these explanations, these are exactly what I was after.