How can I fill a numpy array with empty builtin lists?

Question

I'm filling a numpy grid with lists of unknown size because list.append is better to use than np.concatenate (perhaps I'm optimizing prematurely, though).

These don't work:

foo = np.full((3, 4), []) 
bar = np.full((3, 4), [], dtype=object)

Numpy just assumes that my [] is a numpy array for populating the result and is the wrong shape.

It works for dicts:

foo = np.full((3, 4), {})

Is there some way to do this?

Edit: actually, I don't know how to do this even with numpy arrays. I want a 2x2 grid with variable length lists or arrays in each cell.

Edit 2: what I really want is something like scipy.stats.binned_statistic_2d but instead of a statistic in each bin, I want to keep the source data.

Another problem with full and mutable objects is it puts a reference to the same object in each slot, so you end up with the same sort problems as with making a list of lists with [{}]*3 — hpaulj
– hpaulj, Commented Dec 13, 2019 at 21:34
using a numpy.ndarray of list objects makes no sense. An nd.array of dtype=object is pretty much just a crappy, less efficient list. Just use a list of lists. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Dec 13, 2019 at 21:41
I accepted a good answer below, @juanpa.arrivillaga , but I think you may be right — Jim Hunziker
– Jim Hunziker, Commented Dec 13, 2019 at 21:53
I’ll second what @juanpa.arrivillaga said. I’m disappointed that the accepted answer didn’t even mention it. If you can share a bit more about what you need this for, we might be able to find a better alternative. — AMC
– AMC, Commented Dec 13, 2019 at 22:42
I think the accepted answer is the correct answer to my question. But what I ended up doing was making a list of lists of lists, using np.digitize to determine grid destinations for objects that each have histories of x and y positions, and appending a timestamp+object reference pair to each grid destination list. Basically, I'm trying to put (time, x, y) objects in a 2d grid, and my source data is sparse both in timestamp and position. I then do postprocessing on these grid lists to create representative summary data. — Jim Hunziker
– Jim Hunziker, Commented Dec 13, 2019 at 22:54

hpaulj · Accepted Answer · 2019-12-13 21:40:25Z

full with a mutable object, even when it works has problems:

In [10]: A = np.full(3,{})                                                      
In [11]: A                                                                      
Out[11]: array([{}, {}, {}], dtype=object)
In [12]: A[0]['k']=2                                                            
In [13]: A                                                                      
Out[13]: array([{'k': 2}, {'k': 2}, {'k': 2}], dtype=object)

Like

In [14]: [{}]*3                                                                 
Out[14]: [{}, {}, {}]

it's putting the same object in each slot.

You could create an 'blank' object dtype array, and fill it from a list - taking care that each element is a 'fresh' list:

In [15]: A = np.empty((2,2),object)                                             
In [16]: A                                                                      
Out[16]: 
array([[None, None],
       [None, None]], dtype=object)
In [17]: A[:] = [[[] for _ in range(2)] for _ in range(2)]                      
In [18]: A                                                                      
Out[18]: 
array([[list([]), list([])],
       [list([]), list([])]], dtype=object)
In [19]: A[0,0].append(34)                                                      
In [20]: A                                                                      
Out[20]: 
array([[list([34]), list([])],
       [list([]), list([])]], dtype=object)

frompyfunc is a good tool for creating object dtype arrays, filled with all sorts of object classes:

In [23]: B = np.frompyfunc(lambda x:[],1,1)(np.zeros((2,2)))                    
In [24]: B                                                                      
Out[24]: 
array([[list([]), list([])],
       [list([]), list([])]], dtype=object)
In [25]: B[0,0].append(34)                                                      
In [26]: B                                                                      
Out[26]: 
array([[list([34]), list([])],
       [list([]), list([])]], dtype=object)

This too is creating a 'fresh' list for each element.

Collectives™ on Stack Overflow

How can I fill a numpy array with empty builtin lists?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related