Declaring a list/vector/array of numpy arrays in Cython

Question

Say I have several arrays, potentially of different sizes:

A0 = rand(3,3)
A1 = rand(4,4)

In Cython, I can declare their types to get fast item access:

cdef np.ndarray[double, ndim=2] A0
cdef np.ndarray[double, ndim=2] A1

However, say I want to access them by index:

A = (A0,A1)
A[0][2,1] += A[1][1,0]

However, now Cython doesn't know the type of A[0] and A[1], which makes access slow. I don't think Cython has the concept of a "typed tuple". So how can I declare A (or a similar object) so that I still get fast item access in the above expression?

I suspect the best way is to concatenate them into a 3D array and zero (or nan) pad the places where the sizes don't match. I can't think of an obvious better way (but that doesn't mean it doesn't exist!) — DavidW
– DavidW, Commented Jan 23, 2017 at 17:43

hpaulj · Accepted Answer · 2017-01-23 22:50:22Z

1

Whether this is valid cython is speculative, but inspired by the cython example at the end of:

https://docs.scipy.org/doc/numpy/reference/arrays.nditer.html#putting-the-inner-loop-in-cython

I'd suggest:

cdef foo(aTuple):
    cdef np.ndarray[double, ndim=2] A0
    cdef np.ndarray[double, ndim=2] A1
    A0, A1 = aTuple    # use unpacking
    A0[2,1] += A1[1,0]

https://github.com/cython/cython/blob/master/tests/run/unpack.pyx - is the unpack test pyx.

Or maybe use A0 = aTuple[0] and A1 = aTuple[1]; The idea is to use the array typing where it matters, for its own indexing. The tuple just holds pointers, so the typing doesn't matter.

Another useful page when working with arrays in cython is http://cython.readthedocs.io/en/latest/src/userguide/memoryviews.html

edited Jan 23, 2017 at 22:50

answered Jan 23, 2017 at 22:40

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

cfh Over a year ago

I don't think this helps if I want to have something like A[k] with, say, a loop over k?

hpaulj Over a year ago

Not even with for A0 in aTuple: or A0 = aTuple[k]?

cfh Over a year ago

That misses the point; it means that I have to pay the unboxing cost from a generic Python object to an ndarray every time I want to access it, which is what I want to avoid.

hpaulj Over a year ago

I don't know if that 'unboxing' cost is significant. Every tuple contains pointers to Python objects, whether they are arrays, numbers or strings. The purpose of my suggestion is to let Cython optimize the translation of the array operations. We'd have to do some coding and timings to see whether that matters.

Collectives™ on Stack Overflow

Declaring a list/vector/array of numpy arrays in Cython

1 Answer 1

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Related