Dereferencing The Whole Data Of C_void_p Not Only The First Byte

i have a Question about Pythons ctypes and calling C functions bothering me for a couple days now. I'm working with Python 3.5 and ctypes to wrap a C .dll. I've got a C Function a

Solution 1:

This is one way of doing things. Also, ctypes's official doc: [Python 3.5]: ctypes - A foreign function library for Python.

dll.c:

#include<stdio.h>#include<stdlib.h>#if defined(_WIN32)#  define DLL_EXPORT __declspec(dllexport)#else#  define DLL_EXPORT#endif#define C_TAG "From C"#define PRINT_MSG_0() printf("%s - [%s] (%d) - [%s]\n", C_TAG, __FILE__, __LINE__, __FUNCTION__)#define PRINT_ERR_1S(ARG0) printf("%s: %s\n", C_TAG, ARG0)


DLL_EXPORT inttest(void **pptr, size_t count) {
    PRINT_MSG_0();
    if (!pptr) {
        PRINT_ERR_1S("NULL pointer received");
        return-1;
    }
    if (*pptr) {
        PRINT_ERR_1S("Non NULL inner pointer received");
        return-2;
    }
    unsignedchar *buf = (unsignedchar*)malloc(count);
    for (size_t i = 0; i < count; i++) {
        buf[i] = (i + 1) % 0x100;
    }
    *pptr = buf;
    return0;
}


DLL_EXPORT voiddealloc(void **pptr) {
    PRINT_MSG_0();
    if ((pptr) && (*pptr)) {
        free(*pptr);
        *pptr = NULL;
    }
}

code.py:

import sys
import math
from ctypes import c_ubyte, c_int, c_size_t, c_void_p, \
    POINTER, CDLL, \
    cast


VoidPtrPtr = POINTER(c_void_p)

dll_dll = CDLL("./dll.dll")
test_func = dll_dll.test
test_func.argtypes = [VoidPtrPtr, c_size_t]
test_func.restype = c_int

dealloc_func = dll_dll.dealloc
dealloc_func.argtypes = [c_void_p]

DISPLAY_VALUES_COUNT = 5
FORMAT_STRING_PAT = "    idx: {{:{:d}d}} - value: {{:3d}}"def_get_print_indexes(array_size, values_count):
    if array_size <= 0or values_count <= 1or values_count > array_size:
        raise ValueError("Invalid args")
    yield0if array_size > 1:
        if values_count > 2:
            interval_size = array_size / (values_count - 1)
            for idx inrange(1, values_count - 1):
                yieldint(round(idx * interval_size))
        yield array_size - 1def_print_array_values(array, array_size, values_count=DISPLAY_VALUES_COUNT):
    index_width = math.ceil(math.log10(array_size))
    format_string = FORMAT_STRING_PAT.format(index_width)
    for idx in _get_print_indexes(array_size, values_count):
        print(format_string.format(idx, array.contents.contents[idx]))


defmain():
    sizes = [
        10,
        100,
        500,
        1920 * 1080 * 3,
    ]

    for size in sizes:
        UByteArr = c_ubyte * size
        UByteArrPtr = POINTER(UByteArr)
        UByteArrPtrPtr = POINTER(UByteArrPtr)
        print("\nSize: {:d}".format(size))
        data = UByteArrPtrPtr(UByteArrPtr())
        print("data: {:}, data.contents: {:}".format(data, data.contents))
        #print(addressof(data), addressof(data.contents))
        ptr = cast(data, VoidPtrPtr)
        res = test_func(ptr, size)
        if res < 0:
            print("{:s} returned {:d}. Moving on...\n".format(test_func.__name__, res))
            continueprint("data: {:}, data.contents: {:}".format(data, data.contents))
        _print_array_values(data, size)
        dealloc_func(data)
        print("data: {:}, data.contents: {:}".format(data, data.contents))


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

Notes:

• Since the array allocation place (C or Python) wasn't mentioned, I chose the former option (otherwise, the double pointer (void **) wouldn't make much sense). This adds some complexity to the code (among other things: dealloc - to avoid memory leaks). Using the latter option would "generate" less code, and no need for void** (I'm still wondering why it's needed anyway)
• void* is a generic type with not very much info. That's why in C, in order to populate the individual bytes, I had to cast it to unsigned char *. Same thing applies to Python. The way I expressed void ** in Python is via POINTER(c_void_p)
• A lot of the code is for printing purposes. That includes:
  • _get_print_indexes, which is only used to select 5 (DISPLAY_VALUES_COUNT) equidistant (from index perspective) elements in an array
  • _print_array_values - prints the values

Output:

(py35x64_test) e:\Work\Dev\StackOverflow\q051981858>"c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64

(py35x64_test) e:\Work\Dev\StackOverflow\q051981858>dir /b
code.py
dll.c

(py35x64_test) e:\Work\Dev\StackOverflow\q051981858>cl /nologo dll.c /DDLL  /link /DLL /OUT:dll.dll
dll.c
   Creating library dll.libandobject dll.exp

(py35x64_test) e:\Work\Dev\StackOverflow\q051981858>dir /b
code.py
dll.c
dll.dll
dll.exp
dll.lib
dll.obj

(py35x64_test) e:\Work\Dev\StackOverflow\q051981858>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py
Python 3.5.4 (v3.5.4:3f56838, Aug  82017, 02:17:05) [MSC v.190064 bit (AMD64)] on win32


Size:10data: <__main__.LP_LP_c_ubyte_Array_10 object at 0x000001F8189FBBC8>, data.contents: <__main__.LP_c_ubyte_Array_10 object at 0x000001F8189FBC48>
From C - [dll.c] (16) - [test]
data: <__main__.LP_LP_c_ubyte_Array_10 object at 0x000001F8189FBBC8>, data.contents: <__main__.LP_c_ubyte_Array_10 object at 0x000001F8189FBCC8>
    idx: 0 - value:   1
    idx: 2 - value:   3
    idx: 5 - value:   6
    idx: 8 - value:   9
    idx: 9 - value:  10From C - [dll.c] (35) - [dealloc]
data: <__main__.LP_LP_c_ubyte_Array_10 object at 0x000001F8189FBBC8>, data.contents: <__main__.LP_c_ubyte_Array_10 object at 0x000001F8189FBD48>

Size:100data: <__main__.LP_LP_c_ubyte_Array_100 object at 0x000001F8189FBCC8>, data.contents: <__main__.LP_c_ubyte_Array_100 object at 0x000001F8189FBDC8>
From C - [dll.c] (16) - [test]
data: <__main__.LP_LP_c_ubyte_Array_100 object at 0x000001F8189FBCC8>, data.contents: <__main__.LP_c_ubyte_Array_100 object at 0x000001F8189FBC48>
    idx:  0 - value:   1
    idx: 25 - value:  26
    idx: 50 - value:  51
    idx: 75 - value:  76
    idx: 99 - value: 100From C - [dll.c] (35) - [dealloc]
data: <__main__.LP_LP_c_ubyte_Array_100 object at 0x000001F8189FBCC8>, data.contents: <__main__.LP_c_ubyte_Array_100 object at 0x000001F8189FBE48>

Size:500data: <__main__.LP_LP_c_ubyte_Array_500 object at 0x000001F8189FBC48>, data.contents: <__main__.LP_c_ubyte_Array_500 object at 0x000001F8189FBEC8>
From C - [dll.c] (16) - [test]
data: <__main__.LP_LP_c_ubyte_Array_500 object at 0x000001F8189FBC48>, data.contents: <__main__.LP_c_ubyte_Array_500 object at 0x000001F8189FBDC8>
    idx:   0 - value:   1
    idx: 125 - value: 126
    idx: 250 - value: 251
    idx: 375 - value: 120
    idx: 499 - value: 244From C - [dll.c] (35) - [dealloc]
data: <__main__.LP_LP_c_ubyte_Array_500 object at 0x000001F8189FBC48>, data.contents: <__main__.LP_c_ubyte_Array_500 object at 0x000001F8189FBF48>

Size:6220800data: <__main__.LP_LP_c_ubyte_Array_6220800 object at 0x000001F8189FBDC8>, data.contents: <__main__.LP_c_ubyte_Array_6220800 object at 0x000001F818A62048>
From C - [dll.c] (16) - [test]
data: <__main__.LP_LP_c_ubyte_Array_6220800 object at 0x000001F8189FBDC8>, data.contents: <__main__.LP_c_ubyte_Array_6220800 object at 0x000001F8189FBEC8>
    idx:       0 - value:   1
    idx: 1555200 - value:   1
    idx: 3110400 - value:   1
    idx: 4665600 - value:   1
    idx: 6220799 - value:   0From C - [dll.c] (35) - [dealloc]
data: <__main__.LP_LP_c_ubyte_Array_6220800 object at 0x000001F8189FBDC8>, data.contents: <__main__.LP_c_ubyte_Array_6220800 object at 0x000001F8189FBEC8>

@EDIT0:

• Cleaned the code a bit (some renames, ...)
• Separated printing (added _print_array_values)
• Modified the ctypes pointers to be more meaningful. What stands out and can be confusing, is the relationship between a pointer and an array (when wrapped by another (outer) pointer):
  • In C they are somewhat equivalent: both reference the address of the 1 (array) element. Easily cast one to another
  • In Python the array must be wrapped by POINTER (like a pass by value type) in order to be equivalent to the pointer (and cast from / to it)
• Successfully ran the same code on Lnx