Why Is Numpy List Access Slower Than Vanilla Python?

I was under the impression that numpy would be faster for list operations, but the following example seems to indicate otherwise: import numpy as np import time def ver1(): a

Solution 1:

You're using NumPy wrong. NumPy's efficiency relies on doing as much work as possible in C-level loops instead of interpreted code. When you do

for j in range(40):
    b[j]=a[j]

That's an interpreted loop, with all the intrinsic interpreter overhead and more, because NumPy's indexing logic is way more complex than list indexing, and NumPy needs to create a new element wrapper object on every element retrieval. You're not getting any of the benefits of NumPy when you write code like this.

You need to write the code in such a way that the work happens in C:

b[:] = a

This would also improve the efficiency of the list operation, but it's much more important for NumPy.

Solution 2:

Most of what you are seeing is Python object creation from C native types.

A Python list is, at it's heart, an array of PyObject pointers. When a and b are both Python lists, doing b[i] = a[i] will imply:

• decreasing the reference count of the object pointed by b[i],
• increasing the reference count of the object pointed by a[i], and
• copying the address stored in a[i] into b[i].

But if a and b are NumPy arrays, things are a little more ellaborate, and the same b[i] = a[i] then requires:

• creating a Python integer object from the native C integer type stored at a[i], see this,
• converting the Python integer object into a native C integer type, and storing its value in b[i], see here, and
• decreasing the reference count of the temporary Python integer object.

So the difference is mostly in creating and disposing of that intermediate Python object, that lists do not need to do.