1

I'm working on some high-dimensional integer-based data, and for memory reasons we have to use int8s for everything. The issue I'm running into is that I need to do matrix multiplications with this int8 data, but if the result goes higher than 127, I need it to just return 127 instead of whatever the overflow is.

For example:

import numpy as np

a = np.random.choice([0,1], size=[128,2000]).astype(np.int8)
b = np.random.randint(0,128, size=[2000,2000]).astype(np.int8)

c = np.matmul(a, b)

# c returns values between -128 and 127, instead of the desired 0 to 127

To clarify, I'm not just looking for the extra space unsigned ints would afford - values in b can sometimes be negative. (I just made b all positive to further illustrate my point.)

Is there an algorithm or Numpy trick that would allow me to cap these operations instead of having them overflow? I've perused the Numpy documentation and asked some friends of mine in the CS department, but I've yet to find any leads on this.

Lastly, I know that I could do some of this in C with bit-twiddling, but the rest of our project is unavoidably in Python with little chance of expansion (and given Python, I'd prefer to use Numpy for the matrix multiplication if at all possible).

Thanks, all!

Improve this question
7
  • can you check the definition of a? This does not work as written. Otherwise thanks for the nice verifiable example Commented Oct 24, 2018 at 19:57
  • Rather than insisting on int8 for everything, it's probably going to be much easier to perform this operation in small chunks with a larger dtype. Commented Oct 24, 2018 at 20:02
  • 1
    @kevinkayaks, that's fixed now. Sorry about that! I forgot the brackets for the choice parameter. Commented Oct 24, 2018 at 20:06
  • @user2357112 How would such an operation be performed? Commented Oct 24, 2018 at 20:06
  • @user2357112 Ah, sorry, I understand now. I misinterpreted your statement to be about putting int8s into larger memory spaces, but you actually meant something along the lines of transitioning to int32s and back, right? I did try something similar, actually, but doing that puts me back in trouble with memory requirements. I'm pushing the project right up against our server's memory limits as is, so while I'd love to do the conversion, I don't think I'll be able to. Commented Oct 24, 2018 at 20:20

1 Answer 1

Reset to default
2

Maybe something like this would work for you. This is doing the operation row by row so you only need to hold one row at a time in an int32 datatype. This is a chunking method like @user2357112 is describing. 

def matmul(a, b):
    """chunked matmul which converts datatypes and filters values too large to 127"""
    c = np.empty((a.shape[0], b.shape[1]), dtype = np.int8) # output
    for i in range(a.shape[0]): # iterate over rows in a 
        aa = a[i].astype(np.int32) # convert one row to extended datatype
        cc = aa @ b # broadcasting means cc is the dtype of aa 
        cc[cc > 127] = 127 # set all future overflows to 127
        c[i] = cc.astype(np.int8) # convert dtype back 
    return c

c  = matmul(a, b) # your computation of interest

This will be much slower but may not overload your memory.

If this doesn't work, you can load rows and columns of a and b into memory as needed using np.load with the mmap_mode keyword as explained here. Then you can perform the computation row @ col to develop a single element c[i,j] in the desired datarange.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

That makes a lot of sense. I think extending single rows like may work quite well. Cheers!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.