store/load numpy array from binary files

Question

I would like to store and load numpy arrays from binary files. For that purposes, I created two small functions. Each binary file should contain the dimensionality of the given matrix.

def saveArrayToFile(data, fileName):
    with open(fileName, 'w') as file:
        a = array.array('f')
        nSamples, ndim = data.shape
        a.extend([nSamples, ndim]) # write number of elements and dimensions
        a.fromstring(data.tostring())
        a.tofile(file)


def readArrayFromFile(fileName):
    _featDesc = np.fromfile(fileName, 'f')
    _ndesc = int(_featDesc[0])
    _ndim  = int(_featDesc[1])
    _featDesc = _featDesc[2:]
    _featDesc = _featDesc.reshape([_ndesc, _ndim])

    return _featDesc, _ndesc, _ndim

An example on how to use the functions is:

myarr=np.array([[7, 4],[3, 9],[1, 3]])
saveArrayToFile(myarr,'myfile.txt')
_featDesc, _ndesc, _ndim = readArrayFromFile('myfile.txt')

However, an error message of 'ValueError: total size of new array must be unchanged' is shown. My arrays can be of size MxN and MxM. Any suggestions are more than welcomed. I think the problem might be in the saveArrayToFile function.

Best wishes,

Javier

unutbu · Accepted Answer · 2010-06-16 11:48:04Z

2

Use numpy.save (and numpy.load) to dump (retrieve) numpy arrays to (from) a binary file.

answered Jun 16, 2010 at 11:48

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Javier Over a year ago

OK, thanks! however, I really need to store the MxN dimension of the array into the file.

unutbu Over a year ago

Would it be sufficient to name the file something like myfile_MxN.npy?

unutbu Over a year ago

@Javier: I could understand why you might want to save the dimension of the array in say the first few bytes of the file if you wanted to grab that data without reading the entire file. However, _featDesc, _ndesc, _ndim = readArrayFromFile('myfile.txt') suggests you are reading the entire file (no peeking at header bits). So why not simply use arr=np.load(...), and then _ndesc,_ndim = arr.shape?

Collectives™ on Stack Overflow

store/load numpy array from binary files

1 Answer 1

3 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Linked

Related