Deadlock in Windows XP

Apr 27, 2015 at 12:46 PM
If cpprest compiled with CPPREST_FORCE_PPLX and CPPREST_TARGET_XP macroses application can be catch the deadlock in Windows XP. This sample can easy demonstrate deadlock only in Windows XP:
#include "stdafx.h"
#include <cpprest/http_client.h>
#include <thread>

int _tmain(int argc, _TCHAR* argv[])
{
    std::cout << std::this_thread::get_id() << std::endl;
    pplx::create_task([](){
        std::cout << std::this_thread::get_id() << std::endl;
        pplx::create_task([](){
            std::cout << std::this_thread::get_id() << std::endl;
        }).get();
    }).get();
    return 0;
}
Output:
3768
3104

There is happend due to the fact that the QueueUserWorkItem function using WT_EXECUTEDEFAULT flag. If I am changing it on WT_EXECUTELONGFUNCTION, the deadlock is gone.

Maybe you have more effective solution...

p.s.
I am using CPPREST_FORCE_PPLX because the scheduler of PPL on the Concurrency Runtime has deadlock problem too but more complex. Emulation switching context in crt is bad idea)
I am using CPPREST_TARGET_XP because unfortunately my company has 25% users on Windows XP.

Sorry my english is poor.
Coordinator
Apr 28, 2015 at 5:17 PM
Hi LeonidCSIT,

Yes I see what is happening here, I think this is mostly and issue of running on Windows XP. You only will hit this issue if you use both CPPREST_FORCE_PPLX and CPPREST_TARGET_XP together. Apparently on XP the thread pool won't use another thread unless WT_EXECUTELONGFUNCTION is specified.

I think what you have done might be the best solution. However I'm hesitant to make this change in the code base. If WT_EXECUTELONGFUNCTION is specified for all tasks this could have a negative impact. A new thread could end up being created for each task, which would be inefficient. For small running tasks wouldn't be good.

Please note with Visual Studio 2015 this no longer will be an issue as well because there will be no point to using CPPREST_FORCE_PPLX, as by default all PPL tasks will run on the Windows thread pool instead of the Concurrency Runtime.

Steve
Apr 29, 2015 at 12:16 PM
Thanks Steve!
Yes I agree that for small running tasks it will be overhead and we will be lose time. But I think it is better than catch potential deadlock in production. Unfortunately we catched it(

When we were testing everything works fine. But when we released our software in production some our clients said that the software is doing nothing and they can't work( It happend because as always the deadlock can happened and can't be happened in different situations. For example, code below will be work fine on Windows XP:
#include "stdafx.h"
#include <cpprest/http_client.h>
#include <thread>

int _tmain(int argc, _TCHAR* argv[])
{
    std::cout << std::this_thread::get_id() << std::endl;
    for (int i = 0; i < 10; ++i)
    {
        pplx::create_task([](){
            std::cout << std::this_thread::get_id() << std::endl;
            std::this_thread::sleep_for(std::chrono::seconds(1));
        }).get();
    }
    pplx::create_task([](){
        std::cout << std::this_thread::get_id() << std::endl;
        pplx::create_task([](){
            std::cout << std::this_thread::get_id() << std::endl;
        }).get();
    }).get();
    return 0;
}
Output:
3300
2296
2296
2296
2296
2296
2296
2296
2296
2296
2296
2296
5740

I think lose time better than potential deadlock. May be can you fix potential deadlock in the casablanca and in VS2015? I will be glad if you will do it. If you want to running small tasks in current thread for minimize execution time I would recommend decide run task in current thread or into the free thread in the compile time after analyze the code. It will be difficult to implement but better than now. Or you can do as it did C# like Task.RunSynchronously . Potential deadlock is evil!
Coordinator
Apr 30, 2015 at 10:21 PM
Hi LeonidCSIT,

I've thought about this some more and decided to change it.

Again this only affects when using both CPPREST_FORCE_PPLX and CPPREST_TARGET_XP together.

The fix using WT_EXECUTELONGFUNCTION is in the development branch now and will be in the 2.6.0 release.

Steve
May 7, 2015 at 3:03 PM
Thank you very much Steve! Should I post this problem to Microsoft Connect for fixing this problem on VS2015?
Coordinator
May 7, 2015 at 4:32 PM

Does this cause a problem when not using CPPREST_FORCE_PPLX?

Steve

May 7, 2015 at 9:37 PM
Edited May 7, 2015 at 9:47 PM
I am sorry I thought that tasks that enable "CPPREST_FORCE_PPLX" using in new Visual Studio 2015.

I am worrying about that because VS2013 PPL tasks are so complex and it is so difficult to describe problem of them. But I try to describe one problem here in sample code below.
#include "stdafx.h"
#include <ppl.h>
#include <windows.h>
#include <ppltasks.h>
#include <iostream>
#include <vector>
#include <thread>
#include <mutex>

using namespace Concurrency;
using namespace std;

int main(int argc, char* argv[])
{
    size_t concurentThreadsSupported = std::thread::hardware_concurrency();
    cout << concurentThreadsSupported << endl;
    //deadlock hazard increased with concurentThreadsSupported decreasing

    size_t taskAmountForWasteVirtualCores = concurentThreadsSupported - 1;//must be equal to (virtual processor thread amount from Concurrency::IResourceManager) - 1
    vector<task<void>> t;
    HANDLE event = CreateEventA(nullptr, 1, 0, "I am a bad guy");
    for (size_t i = 0; i < taskAmountForWasteVirtualCores; ++i)
        t.push_back(create_task([event]{
        WaitForSingleObject(event, INFINITE);// or sql transaction
    }));
    Sleep(5000); // wait start tasks
    auto locked = create_task([event]{
        cout << "unlocker" << endl;
    Concurrency::wait(100); // or another Concurrency call (switch context)
    SetEvent(event); // commit sql transaction (for example: release table lock)
    cout << "~unlocker ok" << endl;
    });
    auto locker = create_task([event]{
        cout << "locked" << endl;
    WaitForSingleObject(event, INFINITE); // or sql transaction
    cout << "~locked ok" << endl;
    });
    WaitForSingleObject(event, INFINITE); // or sql transaction
    locked.get();
    locker.get();
    for (auto &tt : t)
    {
        tt.get();
    }
    return 0;
}

This sample doesn't have the deadlock in VS2015 PPL tasks.

Do you know that change in VS2015 PPL tasks?
Coordinator
May 8, 2015 at 1:48 AM
Hi LeonidCSIT,

I don't completely understand/follow what you are asking. The CPPREST_FORCE_PPLX macro should only be necessary with VS2013 if you want the tasks to run on the Windows threadpool. Are you seeing issues otherwise?

Steve
May 8, 2015 at 7:26 AM
Hi Steve!
No, I am not. I am sorry it was offtopic. I just wanted to say that I thought that the new casablanca tasks come from Visual Studio 2015. And I told of my worrying about what will be happened when we will decide to migrate to Visual Studio 2015. In my offtopic I demonstrated one problem of 2013 PPL Tasks. And because you are from Microsoft, I hoped that you can describe me how Microsoft fixed the problem of 2013 PPL Tasks into 2015 PPL Tasks.

p.s.
If you would like to discuss with me about the 2013 PPL Tasks and the 2015 PPL Tasks, you can send my a PM or make another thread for it. I think, this topic must be closed :)

Sorry again, my English is poor :( I hope you understand me now.

Thank you Steve!
Coordinator
May 11, 2015 at 6:50 PM
Hi LeonidCSIT,

I tried to contact you through CodePlex but you are listed as being preferred not to contact so I can't. Can you please contact me through codeplex or directly email me at stgates at Microsoft dot com so we can discuss further?

Thanks,
Steve