I'm simply trying to use a masked array to filter out some nanentries.
import numpy as np
# x = [nan, -0.35, nan]
x = np.ma.masked_equal(x, np.nan)
print x
This outputs the following:
masked_array(data = [ nan -0.33557216 nan],
mask = False,
fill_value = nan)
Calling np.isnan() on x returns the correct boolean array, but the mask just doesn't seem to work. Why would my mask not be working as I expect?
You can use np.ma.masked_invalid:
import numpy as np
x = [np.nan, 3.14, np.nan]
mx = np.ma.masked_invalid(x)
print(repr(mx))
# masked_array(data = [-- 3.14 --],
# mask = [ True False True],
# fill_value = 1e+20)
Alternatively, use np.isnan(x) as the mask= parameter to np.ma.masked_array:
print(repr(np.ma.masked_array(x, np.isnan(x))))
# masked_array(data = [-- 3.14 --],
# mask = [ True False True],
# fill_value = 1e+20)
Why doesn't your original approach work? Because, rather counterintuitively, NaN is not equal to NaN!
print(np.nan == np.nan)
# False
This is actually part of the IEEE-754 definition of NaN
Here is another alternative without using mask:
import numpy as np
#x = [nan, -0.35, nan]
xmask=x[np.logical_not(np.isnan(x))]
print(xmask)
Result:
array([-0.35])
np.logical_not is needed here instead of the built-in not because the latter does not broadcast.
np.isnan() returns a boolean array, and as such using x[~np.isnan(x)] would suffice. np.logical_not() is useful for non-boolean array use-cases.
np.ma.masked_invalid().