0

I have an array with values from 0-5, and I want to use numpy.where() to get the indices of where the item equals to 1, but it returns an empty array.

Code:

hf = h5py.File(PATH+'/data.h5', 'r')
data = hf['data']                    #dtype int32 [0,1,2,3,4,2,3,1,1,1,4,5,6]
print(data[1] == 1)                  #prints True
indices = np.where(data == 1)[0]     #ERROR here - returns empty array

1 Answer 1

1

You have to download the dataset to perform tests like this on it.

Using a test file I have hanging around:

In [318]: f = h5py.File('data.h5')
In [319]: list(f.keys())
Out[319]: ['dset', 'dset1', 'vset']
In [320]: f['dset']
Out[320]: <HDF5 dataset "dset": shape (3, 5), type "<f8">

I can index and test a single item, or slice of the dataset

In [321]: f['dset'][1]
Out[321]: array([ 1.,  1.,  1.,  1.,  1.])
In [322]: f['dset'].shape
Out[322]: (3, 5)
In [323]: f['dset'][...]
Out[323]: 
array([[ 1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.]])

But a boolean test on the dataset does not work:

In [324]: f['dset']>0
...
TypeError: unorderable types: Dataset() > int()

==1 works, but compares the dataset objects with 1, and inevitably returns False. That's why where gives you an empty result:

In [325]: f['dset']==1
Out[325]: False

To do the element by element test I have to 'index' the dataset:

In [326]: f['dset'][...]>0
Out[326]: 
array([[ True,  True,  True,  True,  True],
       [ True,  True,  True,  True,  True],
       [ True,  True,  True,  True,  True]], dtype=bool)
Sign up to request clarification or add additional context in comments.

3 Comments

I solved it by changing it to: data = hf['data'] [::]. Thank you :) I have to wait a couple mins before I can accept your answer.
I believe you can also call np.array on the HDF5 dataset object to create a numpy array.
I'd recommend f['dset'].value over np.array(f['dset']). In any case we need to be aware that there is a difference between the dataset and the array in memory.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.