A problem with a proxy

May 31, 2013 at 4:44 PM
Edited Jun 1, 2013 at 11:10 PM
In the office we are behind the proxy.

I would like to understand why the following code works just fine, while the code without the explicit proxy specification times out:
int _tmain(int argc, _TCHAR* argv[])
{
   TestMS();

   std::cout << "Type <any key><ENTER> to exit" << std::endl;
   char inp;
   std::cin >> inp;
   return 0;
}

void TestMS()
{
   http_client_config client_config;
   web_proxy wp(U("http://myofficeproxy:9090"));
   client_config.set_proxy(wp);

   http_client client(U("http://www.microsoft.com"), client_config);
   client.request(methods::GET).then( [=](http_response response) 
   { 
      std::wostringstream ss;
      ss << L"Server returned returned status code " << response.status_code() << L"." << std::endl;
      ss << response.extract_string().get() << std::endl;
      std::wcout << ss.str();
   }).wait();
}
If I delete proxy-related code above and use default http_client_config instead, GET request times out and I get the http_exception saying "The operation timed out".

My understanding was that client configuration default auto proxy setting would not require explicit proxy specification on each of my potential users machines. In fact, if I am forced to require that, it is a big problem for software distribution.

Executing the same code from my machine at home (which is not behind a proxy) works fine and prints html of microsoft.com page, as expected.

I remember somebody from Casablanca team writing once in a Forum, something along the lines "we at Microsoft are also behind a proxy and we don't need to explicitly specify it".

So I wonder what is the deal with my office proxy?
There are no proxy problems when communicating with web servers inside company's domain.

This is 0.6 compiled for VS2010, 32 bit app on Win7 64 bit.
Jun 1, 2013 at 3:00 PM
Edited Jun 2, 2013 at 2:25 AM
Found the last relevant reference to proxy problem:

Old Casablanca Forum post link

Apparently, there were no enhancements done since this was posted in old MSDN Forum?

So what is the recommended method to programmatically discover proxy configuration?
The WinHttpGetDefaultProxyConfiguration function retrieves the default WinHTTP proxy configuration from the registry.
This doesn't seem to work for 32 bit app on 64 bit machine. I get back null strings and the flag with value 1 (WINHTTP_ACCESS_TYPE_NO_PROXY), as if there is no proxy, when in fact there is one.

This function is apparently not using KEY_WOW64_64KEY flag when reading the Registry?!

WinHttpGetIEProxyConfigForCurrentUser works (strings are not null) but there is no guarantee that IE proxy and WinHttp proxy settings are the same.

I can find WinHttpSettings binary value in the Registry (with KEY_WOW64_64KEY flag set), but not any specification on how to interpret those bytes.
Jun 3, 2013 at 2:24 PM
Tonko,

You are right -- we did not spend as much time on proxy functionality as we had planned in this release. Sometimes, plans change, and we spent a couple of months focusing on getting Casablanca into the next version of Visual Studio, and then we found that so much time had passed since the February release that we needed to get something out in May. It did not necessarily have everything we had originally planned for.


Niklas
Coordinator
Jun 3, 2013 at 6:58 PM
Hi Tonko,

By default the http_client (for desktop applications) uses WINHTTP_ACCESS_TYPE_DEFAULT_PROXY with WinHttp. This would pick up settings that are configured on the machine using netsh. If you want auto discovery to be enabled you need to opt in. Try changing the construction of your web_proxy in your code above to the following:
web_proxy wp(web_proxy::use_auto_discovery);
There can be performance costs associated with auto discovery so that is why we have it off by default. Let me know if this works for you.
Thanks,
Steve
Jun 3, 2013 at 10:42 PM
Hi Steve,

my problem is that I can see proxy configuration when typing:
netsh winhttp show proxy
however, the code in the first post doesn't work without the line:
 web_proxy wp(U("http://myofficeproxy:9090"));
Using:
web_proxy wp(web_proxy::use_auto_discovery);
doesn't help either, unfortunately.

My additional finding, that probably explains it, is that the following simple program behaves differently when compiled as 64 bit or as a 32 bit executable, then run on 64 bit Windows 7 desktop:
#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <Winhttp.h>

int _tmain(int argc, _TCHAR* argv[])
{
    WINHTTP_PROXY_INFO info_;
    auto res = WinHttpGetDefaultProxyConfiguration(&info_);
    return 0;
}
When built as a 64 bit app, info_ structure contains non-NULL lpszProxy string and dwAccessType has value 3 (WINHTTP_ACCESS_TYPE_NAMED_PROXY).
When built as a 32 bit app, proxy string is NULL and dwAccessType is 1 (WINHTTP_ACCESS_TYPE_NO_PROXY).

Since this function is documented as returning Registry settings, I could confirm the same by looking into 'regular' and Wow6432Node parts of the Registry, respectively. The content of WinHttpSettings value definitely differs in two cases.

This is the case with all desktops in my office, which is not surprising as they were set up by the IT department, not by developers themselves. Thus, I would expect this not to be such an uncommon occurrence 'out there in the wild'.
Coordinator
Jun 4, 2013 at 6:58 PM
Hi Tonko,

Correct, the settings for the default proxy configuration with netsh can be set different for 32bit and 64bit on pre Windows 8, I know this is confusing :). Here is a blog post explaining. To set 64bit configuration simply open any command prompt and use netsh. For 32bit you need to use the netsh under the Windows\SysWOW64 directory.

It sounds like your proxy settings are configured for 64bit applications. Can you try your example casablanca program on 64bit and see if it works?

Thanks, Steve
Jun 4, 2013 at 9:52 PM
Steve,

Yes, after also building Casablanca for x64 platform, my sample program works without any explicit proxy setup required.
Jul 9, 2013 at 5:38 AM
Hi Tonko,
       Did you use web_proxy::use_auto_discovery or default proxy to solve the problem finally? 
I have a similar problem, I am using Casablanca in Windows 8 Desktop app. Without a proxy(from my home network), it works without any problem. But it fails intermittently from my office network which is behind a proxy.

When I use web_proxy wp(web_proxy::use_auto_discovery), it seems to take a lot of time and shows "operation timed out" error from my home n/w.

My code:
return pplx::create_task([=]
    {
        //web_proxy wp(web_proxy::use_auto_discovery);
        http_client_config config;
        //setting timeout to 3 minute
        config.set_timeout(utility::seconds(180));
        //config.set_proxy(wp);
        http_client client(url,config);
        http_request requestObj(methods::PUT);
        requestObj.headers().add(L"Authorization", L"Bearer "+access_token);
        requestObj.headers().add(header_names::accept, L"application/json");
        requestObj.set_body(putData,L"application/json");
        return client.request(requestObj);
    }).then([](http_response responseObj)
    {
        return responseObj;
    });
Thanks/Sony
Jul 9, 2013 at 3:28 PM
Edited Jul 9, 2013 at 4:26 PM
Hi Sony,

I have the same issue with auto proxy (apparently, it is simply not configured for my office network) but I still give it a try, because it may be set up for other users.
I also try reading the Registry (in the case proxy was configured via netsh) and, if everything else fails, I look for IE proxy configuration.

In theory, at least according to my understanding, all of these three configurations can be different. However, if that is the case, the network admin in question must have very special reasons and I would first talk to her/him, before doing anything.

On the other hand, you can count yourself lucky that you are on Windows 8. According to post by Steve above, if proxy was configured via netsh, it will be read correctly by both 32-bit and 64-bit apps on Windows 8. In my case IE proxy configuration is the same as 64-bit netsh configuration, so ultimately I get the correct proxy.

To read all three possible configurations I use WinHttp API. Once I get the golden proxy string (which strictly speaking is not really URL, because protocol part is missing), I use Casablanca to set up the proxy object for http_client_config.

Example:
bool ProxyConfig::AutoProxyFromUrl(string_t const& siteurl)
{
    if(handle_)
    {
        WINHTTP_AUTOPROXY_OPTIONS  options = {0};
        options.dwFlags = WINHTTP_AUTOPROXY_AUTO_DETECT;
        options.dwAutoDetectFlags = WINHTTP_AUTO_DETECT_TYPE_DHCP|WINHTTP_AUTO_DETECT_TYPE_DNS_A;
        options.fAutoLogonIfChallenged = TRUE;

        if(WinHttpGetProxyForUrl(handle_,
                                 siteurl.c_str(),
                                 &options,
                                 &info_))
        {
            proxyurl_ = info_.lpszProxy;
            return true;
        }
    }
    return false;
}
        auto res = WinHttpGetDefaultProxyConfiguration(&info_);
        if(res == ERROR_SUCCESS && info_.dwAccessType == WINHTTP_ACCESS_TYPE_NAMED_PROXY)
        {
            found = true;
            proxyurl_ = info_.lpszProxy;
        }
bool ProxyConfig::ProxyFromIE()
{
    if(WinHttpGetIEProxyConfigForCurrentUser(&ieinfo_))
    {
        proxyurl_ = ieinfo_.lpszProxy;
        return true;
    }
    return false;
}
Jul 15, 2013 at 4:15 AM
Thanks Tonko. As you said, getting the IE proxy config for the current user did the trick! Our QA has not yet reported any issues after implementing this solution, all is well so far ;-)

Thanks/Sony