function taking ownership of Numpy array

Question

I have a function takes_ownership() that performs an operation on a Numpy array a in place and effectively relies on becoming the owner of the data. Is there a way to alter the passed array such that it no longer points to its original buffer?

The motivation is a code that processes huge chunks of data in place (due to performance constraints) and a common error is unaware users recycling their input arrays.

Note that the question is very specifically not "why is a different if I change b = a in my function". The question is "how can I make using the given array in place safer for unsuspecting users" when I must not make a copy.

def takes_ownership(a):
    b = a

    # looking for this step
    a.set_internal_buffer([])

    # if this were C++ std::vectors, I would be looking for
    # b = np.array([])
    # a.swap(b)

    # an expensive operation that invalidates a
    b.resize((6, 6), refcheck=False)

    # outside references to a no longer valid
    return b


a = np.random.randn(5, 5)

b = takes_ownership(a)

# array no longer has data so that users cannot mess up
assert a.shape = ()

Checking back on this question an hour later, I don't believe there is a clean way to do what you want. I don't have experience with numpy, but this appears to answer your question. Basically, have your arrays in a wrapper, then exchange out the arrays in the wrapper. Numpy doesn't appear to expose a way of simply swapping the underlying structure it holds. — Carcigenicate
– Carcigenicate, Commented Jun 10, 2021 at 21:38

generic_stackoverflow_user · Accepted Answer · 2021-06-10 20:37:11Z

-1

NumPy has a copy function that will clone an array (although if this is an object array, i.e. not primitive, there might still be nested object references after cloning). That being said, this is a questionable design pattern and it would probably be better practice to rewrite your code in a way that does not rely on this condition.

Edit: If you can't copy the array, you will probably need to make sure that none of your other code modifies it in place (instead running immutable operations to produce new arrays). Doing things this way may cause more issues down the road, so I'd recommend refactoring so that it is not necessary.

answered Jun 10, 2021 at 20:37

generic_stackoverflow_user

3264 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ntessore Over a year ago

The question is precisely how to "make sure that none of your other code modifies it".

generic_stackoverflow_user Over a year ago

Right, but you cannot operate on a "different" array unless you (a) copy the whole array or (b) use an immutable operation that returns a new array. If you copy the array correctly, I can't see it being a major detractor from your program's performance - still, as Carcigenicate said, this is a rather strange way of implementing whatever functionality you are trying to design.

Collectives™ on Stack Overflow

function taking ownership of Numpy array

1 Answer 1

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Linked

Related