2

I am looking for a operation witch converts my byte array:

mem = b'\x01\x02\xff'

in something like this:

[ [0 0 0 0 0 0 0 1]
  [0 0 0 0 0 0 1 0]
  [1 1 1 1 1 1 1 1] ]

These are operations that I tried:

import numpy as np

mem = b'\x01\x02\xff' #define my input
mem = np.fromstring(mem, dtype=np.uint8) #first convert to int

#print(mem) give me "[  1   2 255]" at this piont

mem = np.array(['{0:08b}'.format(mem[b]) for b in mem]) #now convert to bin
data= np.array([list(mem[b]) for b in mem]) #finally convert to single bits

print(data)

This code will crash at line 4.. IndexError: index 255 is out of bounds for axis 0 with size 9 Otherwise, it crash at line 5.. IndexError: too many indices for array

These are my Questions:

Why are the number of spaces different after the conversion from hex to int?

Is that the reason that my next conversion from int to bin failed?

Finally, what is wrong with my list operation?

Thank you for your help! :)

Improve this question
1
  • Note that mem[b] should be b in the 2 statements: mem = np.array(...) and data= np.array(...). Commented Sep 27, 2016 at 7:40

4 Answers 4

Reset to default
2

Using unpackbits:

>>> import numpy as np
>>> mem = b'\x01\x02\xff'
>>> x = np.fromstring(mem, dtype=np.uint8)
>>> np.unpackbits(x).reshape(3,8)
array([[0, 0, 0, 0, 0, 0, 0, 1],
       [0, 0, 0, 0, 0, 0, 1, 0],
       [1, 1, 1, 1, 1, 1, 1, 1]], dtype=uint8)

Documentation

From help(np.unpackbits):

unpackbits(...)
unpackbits(myarray, axis=None)

Unpacks elements of a uint8 array into a binary-valued output array.

Each element of myarray represents a bit-field that should be unpacked into a binary-valued output array. The shape of the output array is either 1-D (if axis is None) or the same shape as the input array with unpacking done along the axis specified.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

This is a unbelievable fast operation!! Great! I need this conversion for a huge amount of data and the results are now fantastic. My way to convert the data needs 100 times longer. God to know, that ´unpackbits´ works also well.
0

To solve IndexError you can use numpy.ndindex:

import numpy as np

mem = b'\x01\x02\xff' #define my input
mem = np.fromstring(mem, dtype=np.uint8) #first convert to int

#print(mem) give me "[  1   2 255]" at this piont
mem=np.array(['{0:07b}'.format(mem[b]) for b in np.ndindex(mem.shape)])

data= np.array([list(mem[b]) for b in np.ndindex(mem.shape)]) #finally convert to single bits

print(data)

Output:

[['0', '0', '0', '0', '0', '0', '1'] ['0', '0', '0', '0', '0', '1', '0']
 ['1', '1', '1', '1', '1', '1', '1', '1']]
Improve this answer

1 Comment

Great! Have you any idea what the different numbers of spaces means? (line 6 in your code)
0

I'm fairly certain the problem with your code is that you're assuming the int in each item in the list will become 8 bits (so 2 will, in your assumption, return 00000010). But it doesn't (2 = 10), and that screws up your code.

For your last two lines, I think this should be fine:

data = [list(str(bin(x))[2:]) for x in mem]
for a in range(len(data)):
    while len(data[a]) < 8:
        data[a] = "0" + data[a]

str(bin(x))[2:] converts to binary (because it returns 0b1 for 1, you need to use [2:] to get 1).

The last chunk of code is to "pad" out your numbers with extra 0's.

Improve this answer

Comments

0 
mem = b'\x01\x02\xff'
[[int(digit) for digit in "{0:08b}".format(byte)] for byte in mem]

outputs:

[[0, 0, 0, 0, 0, 0, 0, 1], [0, 0, 0, 0, 0, 0, 1, 0], [1, 1, 1, 1, 1, 1, 1, 1]]
Improve this answer

1 Comment

Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.