Read a float binary file into 2D arrays in python and matlab

Question

I have some binary input files (extension ".bin") that describe a 2D field of ocean depth and which are all negative float numbers. I have been able to load them in matlab as follows:

f = fopen(filename,'r','b');
data = reshape(fread(f,'float32'),[128 64]);

This matlab code gives me double values between 0 and -5200. However, when I try to the same in Python, I strangely get values between 0 and 1e-37. The Python code is:

f = open(filename, 'rb')
data = np.fromfile(f, np.float32)
data.shape = (64,128)

The strange thing is that there is a mask value of 0 for land which shows up in the right places in the (64,128) array in both cases. It seems to just be the magnitude and sign of the numpy.float32 values that are off.

What am I doing wrong in the Python code?

Instead of just doing your own file-parsing, can't you use a file format that's supported by NumPy and Matlab? — MSeifert
– MSeifert, Commented Jun 2, 2017 at 19:11
The file format is ".bin" and unfortunately this is the required format for input files for the model I'm using. Not sure if that answers your question. — henrifdrake
– henrifdrake, Commented Jun 2, 2017 at 19:15

MSeifert · Accepted Answer · 2017-06-02 19:20:05Z

7

numpy.fromfile isn't platform independant, especially the "byte-order" is mentioned in the documentation:

Do not rely on the combination of tofile and fromfile for data storage, as the binary files generated are are not platform independent. In particular, no byte-order or data-type information is saved.

You could try:

data = np.fromfile(f, '>f4')  # big-endian float32

and:

data = np.fromfile(f, '<f4')  # little-endian float32

and check which one (big endian or little endian) gives the correct values.

answered Jun 2, 2017 at 19:20

MSeifert

154k41 gold badges356 silver badges377 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

henrifdrake Over a year ago

Thank you! As noted by @Xiangrui Li, the 'b' argument in the matlab function specifies big-endian.

MSeifert Over a year ago

Ah, good to know. I thought it was equivalent to the 'b' in pythons open (there it stands for "binary"). :)

henrifdrake Over a year ago

That was my thought as well!

Xiangrui Li · Accepted Answer · 2017-06-02 19:17:23Z

0

Base on your matlab fopen, the file is in big endian ('b'). But your python code does not take care of the endianness.

answered Jun 2, 2017 at 19:17

Xiangrui Li

2,4462 gold badges16 silver badges17 bronze badges

Collectives™ on Stack Overflow

Read a float binary file into 2D arrays in python and matlab

2 Answers 2

3 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Related