Get Max/Min from array based on another array using Numpy.where

Question

Starting with this:

import numpy as np
x = np.array([0,   2,  8,  9,  4,    1, 12,  4, 33, 11,    5,  3 ])
y = np.array(['', '', '', '', '', 'yo', '', '', '', '', 'yo', '' ])
i = np.array([0,   1,  2,  3,  4,    5,  6,  7,  8,  9,   10, 11 ])
print np.amax(x[:3] )         
print np.amin(x[:3] )        

Trying to get the max or min value for the prior three items using numpy.where. So, in essence trying to use the "index" of array within the np.where. If there a more performant way to do this, please show.

Tried variations on this:

np.where(y == "yo", np.amax(x[:3] ) ,"")

result (why is it returning a string?):

array(['', '', '', '', '', '8', '', '', '', '', '8', ''], 
      dtype='|S21')

wanted:

 array(['', '', '', '', '', 9, '', '', '', '', 33, ''], 
      dtype='|S21')

Answer 1

First look at the simpler version of where, which finds the indices:

In [266]: np.where(y=='yo')
Out[266]: (array([ 5, 10], dtype=int32),)

Evidently you want all the valyes for y, but replacing the yo with some value from x:

In [267]: np.where(y=='yo',x,y)
Out[267]: 
array(['', '', '', '', '', '1', '', '', '', '', '5', ''], 
      dtype='<U11')

y is string type, and since '' can't be converted to a number, the numbers are converted to string.

Now if y was object dtype:

In [268]: y = np.array(['', '', '', '', '', 'yo', '', '', '', '', 'yo', '' ],object)
In [269]: np.where(y=='yo')
Out[269]: (array([ 5, 10], dtype=int32),)
In [270]: np.where(y=='yo',x,y)
Out[270]: array(['', '', '', '', '', 1, '', '', '', '', 5, ''], dtype=object)

the replacement is also object dtype and can have a mix of numbers and strings.

In this use all 3 terms have the same length. In your use, x and y are replaced with scalars

In [271]: np.max(x[:3])
Out[271]: 8
In [272]: np.where(y=='yo',8, '')
Out[272]: 
array(['', '', '', '', '', '8', '', '', '', '', '8', ''], 
      dtype='<U11')
In [273]: np.where(y=='yo',8, y)
Out[273]: array(['', '', '', '', '', 8, '', '', '', '', 8, ''], dtype=object)

To insert 9 and 33 you have figure out some way of collecting the max of the previous 3 items, i.e. a running or rolling max. where itself isn't going to help.

accumulate approximates this (this is the 'maximum' version of cumsum)

In [276]: xm=np.maximum.accumulate(x)
In [277]: xm
Out[277]: array([ 0,  2,  8,  9,  9,  9, 12, 12, 33, 33, 33, 33], dtype=int32)
In [278]: np.where(y=='yo',xm, y)
Out[278]: array(['', '', '', '', '', 9, '', '', '', '', 33, ''], dtype=object)

xm is not the maximum of the previous three values, but rather the maximum of all previous values. In this case that's the same, but in general it won't. For this x it is different for the last value

---

Here's one way of getting the max of the previous 3, admittedly a bit crude (with a list comprehension):

In [305]: x1=np.concatenate(([0,0],x))
In [306]: xm = [max(x1[i:i+3]) for i in range(0,len(x1))][:len(x)]
In [307]: xm
Out[307]: [0, 2, 8, 9, 9, 9, 12, 12, 33, 33, 33, 11]
In [308]: np.where(y=='yo',xm, y)
Out[308]: array(['', '', '', '', '', 9, '', '', '', '', 33, ''], dtype=object)

---

sliding window with as_strided (adapted from Numpy: Matrix Array Shift / Insert by Index)

In [317]: xm=np.lib.stride_tricks.as_strided(x1[::-1],shape=(3,12),strides=(-4,-4))
In [318]: xm
Out[318]: 
array([[ 3,  5, 11, 33,  4, 12,  1,  4,  9,  8,  2,  0],
       [ 5, 11, 33,  4, 12,  1,  4,  9,  8,  2,  0,  0],
       [11, 33,  4, 12,  1,  4,  9,  8,  2,  0,  0,  0]])
In [319]: xm.max(axis=0)
Out[319]: array([11, 33, 33, 33, 12, 12,  9,  9,  9,  8,  2,  0])
In [320]: xm = xm.max(axis=0)[::-1]
In [321]: xm
Out[321]: array([ 0,  2,  8,  9,  9,  9, 12, 12, 33, 33, 33, 11])

---

Using Paul Panzer's idea for just a few yo:

In [29]: idx=np.where(y=='yo')
In [30]: idx
Out[30]: (array([ 5, 10], dtype=int32),)

In [32]: xm = [max(x[i-3:i]) for i in idx[0]]
In [33]: xm
Out[33]: [9, 33]
In [34]: y[idx]=xm
In [35]: y
Out[35]: array(['', '', '', '', '', 9, '', '', '', '', 33, ''], dtype=object)

If it is possible that yo occurs in the first 3 elements, we need to refine xm with something like:

xm = [max(x[max(i-3,0):i+1]) if i>0 else x[i] for i in idx[0]]

otherwise we get errors from trying to take max([]).

Answer 2

The 'wanted' item I'm afraid you can't have because you can't have numbers in an array of string dtype. where in the form you are using it 'mixes' its last two arguments into one array. For this it has to pick a dtype. It is going for string because of

>>> np.can_cast(str, int)
False
>>> np.can_cast(int, str)
True

so str is the one of the two arguments' dtypes/types that can accomodate values from both arguments.

Data types aside you probably want to have a look at scipy.ndimage.maximum_filter:

>>> scipy.ndimage.maximum_filter(x, 3)
array([ 2,  8,  9,  9,  9, 12, 12, 33, 33, 33, 11,  5])

You may have to fix the offset to suit your reqirements.

Answer 3

Not sure I understand what you want but does this helps :

x = np.sort(x)
sel = np.where(y=="yo")[0]
y[sel] = x[-len(sel):] 

?