How to pass a numpy array of string types to a function in Cython

Question

Passing a numpy array of dtype np.float64_t works fine ( below), but I can't pass string arrays.

This is what works :

# cython_testing.pyx
import numpy as np
cimport numpy as np

ctypedef np.float64_t dtype_t 

cdef func1 (np.ndarray[dtype_t, ndim=2] A):
    print A 

def testing():
    chunk = np.array ( [[94.,3.],[44.,4.]], dtype=np.float64)

    func1 (chunk)

But I can't make this work: I can't find the matching 'type identifiers' for numpy string dtypes.

# cython_testing.pyx
import numpy as np
cimport numpy as np

ctypedef np.string_t dtype_str_t 

cdef func1 (np.ndarray[dtype_str_t, ndim=2] A):
    print A 

def testing():
    chunk = np.array ( [['huh','yea'],['swell','ray']], dtype=np.string_)

    func1 (chunk)

The compilation error is :

Error compiling Cython file:
------------------------------------------------------------
ctypedef np.string_t dtype_str_t 
    ^
------------------------------------------------------------

cython_testing.pyx:9:9: 'string_t' is not a type identifier

UPDATE

Per looking through numpy.pxd, I see the following ctypedef statements. Maybe that's enough to say I can use uint8_t and pretend everything is normal, as long as I can do some casting?

ctypedef unsigned char      npy_uint8
ctypedef npy_uint8      uint8_t

Just have to see how expensive that casting will be.

Saullo G. P. Castro · Accepted Answer · 2014-06-04 20:11:57Z

9

With Cython 0.20.1 it works using cdef np.ndarray, without specifying the data type and the number of dimensions:

import numpy as np
cimport numpy as np

cdef func1(np.ndarray A):
    print A

def testing():
    chunk = np.array([['huh','yea'], ['swell','ray']])
    func1(chunk)

answered Jun 4, 2014 at 20:11

Saullo G. P. Castro

59.4k28 gold badges191 silver badges244 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Saullo G. P. Castro Over a year ago

@TedPetrou I am trying to build an example where the dtype=object would accelerate in order to update the answer, but up to now I found it to be equivalent to not specifying dtype. How did you measure the 100x speed up?

Ted Petrou Over a year ago

Looks like I massively misspoke in my previous comment. It looks like I am getting 5x improvement by changing to object. Use this array. a = np.array(['some', 'strings', 'in', 'an', 'array'] * 10 ** 5)

JAB · Accepted Answer · 2017-03-13 22:24:28Z

7

Looks like you're out of luck.

http://cython.readthedocs.org/en/latest/src/tutorial/numpy.html

Some data types are not yet supported, like boolean arrays and string arrays.

This answer is no longer valid as shown by Saullo Castro's answer, but I'll leave it for historical purposes.

edited Mar 13, 2017 at 22:24

answered Jun 12, 2012 at 20:43

JAB

21.2k6 gold badges73 silver badges80 bronze badges

4 Comments

HeyWatchThis Over a year ago

Thanks. I upvoted your answer. Though I hope there is a work around by using perhaps the Numpy Structured array [docs.scipy.org/doc/numpy/user/…. But I am still looking for how to pass one of those too.

HeyWatchThis Over a year ago

At least for my purposes, using cProfile, it looks like you can still pass Numpy arrays w/o typing, in Cython. But you do not get the Cython optimizations described in your readthedocs.org reference.

JAB Over a year ago

Being able to use them slowly is still better than not being able to use them at all, though, right?

gzc Over a year ago

Content of this link has been modified. The quote doesn't exist.

Collectives™ on Stack Overflow

How to pass a numpy array of string types to a function in Cython

2 Answers 2

2 Comments

4 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

4 Comments

Linked

Related