Numpy - Matrix multiplication

Question

I spent about 2 hours trying to figure out what's going on, but for the life of me I can't figure out why switching the order of matrices when multiplying doesn't seem to work:

Using Python 2.7x:

import numpy as np
num_segments = 25
num_vintages = 24
# Create a 3d matrix 25 deep x 24r x 24c
mx_loans_new_loans = np.zeros((num_segments, num_vintages, num_vintages))

# Create the multiplication vector - same as the first dimension of 3d matrix
mult_vector = np.arange(10,35)
len(mult_vector)

The intent is to fill in the matrix on the diagonal with the mult_vector. Here is the part I can't wrap my head around.

This version does not fulfill the intent:

for (i, x) in enumerate(mx_loans_new_loans):
    np.fill_diagonal(x, 1)
    x = x * mult_vector[i]

The results still spit out just the original matrix with 1's on the diagonal.

This version, however, does work. All I've done is reverse the matrices in the enumerate:

for (i, x) in enumerate(mult_vector):
    np.fill_diagonal(mx_loans_new_loans[i], 1)
    mx_loans_new_loans[i] = mx_loans_new_loans[i] * x

Sidenote: I've since realized a more optimized version fills the intent:

for (i, x) in enumerate(mx_loans_new_loans):
    np.fill_diagonal(x, mult_vector[i])

But does anyone know why the first version doesn't work, but the second version does? What am I missing? Is this a broadcasting problem or something simpler?

Answer 1

mx_loans_new_loans is 3d, (25,24,24). x in the loop is (24,24). multi_vector is (25,); multivector[i] a scalar.

The fill_diagonal sets the 1s in x (in place). But x = x * mult_vector[i], replaces x with a new array, and doesn't change the original. That is, it reassigns variable x.

x *= mult_vector[i]

should change the diagonals in the original array.

mx_loans_new_loans[i] = ... changes this subarray in place. Again mx_loans_new_loans[i] *= ... should work as well.

With zip we can iterate on both arrays:

In [44]: Z=np.zeros((4,3,3),int)
In [45]: for i,j in zip(Z,np.arange(10,14)):
    ...:     np.fill_diagonal(i,j)
    ...:     
In [46]: Z
Out[46]: 
array([[[10,  0,  0],
        [ 0, 10,  0],
        [ 0,  0, 10]],

       [[11,  0,  0],
        [ 0, 11,  0],
        [ 0,  0, 11]],

       [[12,  0,  0],
        [ 0, 12,  0],
        [ 0,  0, 12]],

       [[13,  0,  0],
        [ 0, 13,  0],
        [ 0,  0, 13]]])

We can view all the diagonals as set in the loop with:

In [47]: Z[:,np.arange(3),np.arange(3)]
Out[47]: 
array([[10, 10, 10],
       [11, 11, 11],
       [12, 12, 12],
       [13, 13, 13]])

And modify them with (not quite right):

In [48]: Z[:,np.arange(3),np.arange(3)]=np.arange(20,23)
In [49]: Z
Out[49]: 
array([[[20,  0,  0],
        [ 0, 21,  0],
        [ 0,  0, 22]],

       [[20,  0,  0],
        [ 0, 21,  0],
        [ 0,  0, 22]],

       [[20,  0,  0],
        [ 0, 21,  0],
        [ 0,  0, 22]],

       [[20,  0,  0],
        [ 0, 21,  0],
        [ 0,  0, 22]]])

better:

In [50]: Z[:,np.arange(3),np.arange(3)]=np.arange(20,24)[:,None]
In [51]: Z
Out[51]: 
array([[[20,  0,  0],
        [ 0, 20,  0],
        [ 0,  0, 20]],

       [[21,  0,  0],
        [ 0, 21,  0],
        [ 0,  0, 21]],

       [[22,  0,  0],
        [ 0, 22,  0],
        [ 0,  0, 22]],

       [[23,  0,  0],
        [ 0, 23,  0],
        [ 0,  0, 23]]])

---

Another example of how assignment changes arrays (or not):

In [97]: x = np.zeros((2,3),int)

y is a view, a way of looking at one row of x:

In [99]: y = x[0]
In [100]: id(y)
Out[100]: 2877147400

In-place change to y appears as a change to x:

In [101]: y += 3
In [102]: y
Out[102]: array([3, 3, 3])
In [103]: x
Out[103]: 
array([[3, 3, 3],
       [0, 0, 0]])

But a y= change to y breaks the connection. y is no longer a view (row) of x, but rather a whole new array.

In [104]: y = y + 3
In [105]: y
Out[105]: array([6, 6, 6])
In [106]: x
Out[106]: 
array([[3, 3, 3],
       [0, 0, 0]])
In [107]: id(y)
Out[107]: 2876795264