Deadlock When QThread Tries To Acquire Python GIL Via PyGILState_Ensure()

I have a C++/Qt application in which I want to embed the Python interpreter. I want to call Python from a QThread, but I'm getting a deadlock at the line where I call PyGILState_En

Solution 1:

After browsing around SO, I've found a solution. Example 1 in this answer was especially helpful.

What I need to do is to call PyEval_InitThreads() from the main thread (not at all clear from the very cryptic documentation). Then, to enable PyGILState_Ensure() to acquire the GIL at all from other threads (or else get stuck in an infinite loop within the Python source code, continually trying and failing to acquire the GIL), I need to release the GIL in the main thread via a call to PyEval_SaveThread(). Finally, I should retrieve the GIL in the main thread again via a call to PyEval_RestoreThread() (making sure that all threads that want to call PyGILState_Ensure() are definitely finished before that, or else risking a lock again for the same reason as before).

Here is an updated version of main.cpp which solves the problem:

#include <QCoreApplication>
#include <QThread>
#include "Worker.h"

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    Py_Initialize();

    //Initialize threads and release GIL:
    PyEval_InitThreads();
    PyThreadState *threadState;
    threadState = PyEval_SaveThread();

    QThread* thread = new QThread;
    Worker* worker = new Worker();
    worker->moveToThread(thread);
    QObject::connect(thread, SIGNAL(started()), worker, SLOT(process()));
    QObject::connect(worker, SIGNAL(finished()), thread, SLOT(quit()));
    QObject::connect(worker, SIGNAL(finished()), worker, SLOT(deleteLater()));
    QObject::connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
    thread->start();

    //wait until thread is finished calling Python code:
    thread->wait(1000); //(ugly, but in a proper Qt application, this would be handled differently..)

    //Retrieve GIL again and clean up:
    PyEval_RestoreThread(threadState);
    Py_FinalizeEx();

    return a.exec();
}