Empty response using https on Windows

Jan 10, 2014 at 3:42 PM
Edited Jan 10, 2014 at 3:46 PM
I have a simple piece of test code that looks like this:
    http_client client(L"https://syntagmweb/cgi-bin/sc-validate.pl");

    // Make the request and asynchronously process the response. 
    return client.request(methods::GET).then([](http_response response)
    {
        TRACE(L"Server returned returned status code %d\n", response.status_code());

        // TODO: Perform actions here reading from the response stream.
        auto bodyStream = response.body();

        // In this example, we print the length of the response to the console.
        TRACE(L"Content length is %d\n", response.headers().content_length());
    });
When the URI in the http_client contructor uses 'http', the content length reported is around 2KB. But when the constructor uses 'https' the response length is 0. The response code is 200 in both cases. (The full URI is a local test server.)

Any ideas why no response data for the https version? If I use a browser with the two URIs I get the same results - around 2KB of web page.
Jan 11, 2014 at 11:52 AM
Edited Jan 11, 2014 at 4:10 PM
I have investigated this further using Wireshark. For the http URI the request and response are exactly as expected. With https, no request is being sent on the wire even though I am gettin a response code of 200. What's going on? The data I am proposing to send to the server is not fantastically sensitive but I really should be using https.
Jan 11, 2014 at 2:34 PM
I have changed over to WinHttp and it works fine using the INTERNET_DEFAULT_HTTPS_PORT option to WinHttpConnect. So what is the problem with trying to do this with C++ REST? I am using V1.4 with the RTM version of VS2013. I added the NuGet module for C++ REST to my project. The packages.config file reads <package id="cpprestsdk" version="1.4.0" targetFramework="Native" />.
Coordinator
Jan 15, 2014 at 8:11 PM
Hi whudson_syntagm,

From what you are describing here it sounds like some sort of a caching issue although it seems weird with http vs. https, but I'm not sure. I have a couple of questions to help clarify. What platform are you running on with the C++ Rest SDK? Desktop, Windows 8 store, etc...? Specifically you mention changing over to WinHttp, which is what powers our desktop implementation.

Another possibility is this could be a known bug that we are working on where the server is indicating the length of the message by closing the connection instead of providing a Content-Length header. This also would seem unusual for the behavior to be different from http to https. Could you try capture the entirety of the response with Wireshark or another tool and let me see it? If you want to take this offline to not share any personal data feel free to email me directly, stgates at Microsoft dot com.

Thanks,
Steve
Coordinator
Jan 20, 2014 at 10:07 PM
To circle back around and share on CodePlex William shared with me a code snippet and his test server and we figured out the problem.

I noticed that there isn’t a Content-Length header and Transfer-Encoding: chunked is being used instead to deliver the message body. Next I took a look at your code provided and made a couple of tweaks printing out the whole response message and checking for the existence of the Content-Length header. Here is the code I used:
    http_client client(L"testserver");

    // Make the request and asynchronously process the response. 
    auto response = client.request(methods::GET).get();
    wprintf(L"MSG:%s\n", response.to_string().c_str());

    wprintf(L"****************************\n");
    if(response.headers().has(L"Content-Length"))
    {
        wprintf(L"Has Content-Length header.\n");
    }
    else
    {
        wprintf(L"Missing Content-Length header.\n");
    }
    wprintf(L"Content-Length:%u\n", response.headers().content_length());
Running this gives the following output:

MSG:HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Date: Sat, 18 Jan 2014 00:25:24 GMT
MS-Author-Via: DAV
Server: nginx
Transfer-Encoding: chunked
Vary: Accept-Encoding
X-Powered-By: PleskLin

Sample data

Missing Content-Length header.
Content-Length:0

What is going on is the http_headers::content_length() method will return 0 if there isn’t actually a Content-Length header field present. This was confusing you and leading you to think there was no actual body data, when in fact there was. I think this behavior isn’t idea, but I’m not sure of a better solution. The other option is we could have it throw an exception if the Content-Length header doesn’t exist, but I’m not sure that is so great. If you care about the presence of a specific header it probably best to check with the http_headers::has(…) method.

Please note as well if you don’t care about knowing the actual underlying length of the data you can always just directly read from the response stream until the end is reached or use one of the http_response::extract_* helper methods like extract_string(). In all cases you will always get the data from the body regardless of the method transferred.

Steve