Is There A Good Way To Send Data From Python Context To C++ Without Too Much Copy Involved

The title could be confusing. Here I will state my question more clearly. I want to create a website based on python (a lot of existing framework like Flask and cherryPy) and along

Solution 1:

However, every time I send a data from python, say a matrix, to C++, I have to use python list, which means I have to transform the matrix data into list and in C++ context transform the list into an internal matrix object.

No, you don't have to use the python list. You can use numpy array which allocates data as a C contiguous segment which can be passed down to C++ without copying and viewed as a matrix using a matrix wrapper class.

In python allocate 2d array using numpy:

>>>y=np.empty((2,5), dtype=np.int16)>>>y
array([[   12, 27750, 26465,  2675,     0],
       [    0,     0,     0,  2601,     0]], dtype=int16)
>>>y.flags['C_CONTIGUOUS']
True
>>>foo(y,2,5)

Pass matrix data to C++ using below function exposed to python:

voidfoo(python::object obj, size_t size1, size_t size2){
    PyObject* pobj = obj.ptr();
    Py_buffer pybuf;
    PyObject_GetBuffer(pobj, &pybuf, PyBUF_SIMPLE);
    void *buf = pybuf.buf;
    int16_t *p = (int16_t*)buf;
    Py_XDECREF(pobj);

    MyMatrixWrapper matrix(p, size1, size2);
    // ....
}

Solution 2:

There's a project called ndarray with this as their exact aim: https://github.com/ndarray/ndarray see also https://github.com/ndarray/Boost.NumPy .

There is some compatibility with other C++ matrix libraries (eg. Eigen) which should help with the computations.