Pandas byte array to hex conversion

Question

So I've got a pandas dataframe containing a column of byte array.

byte_array
[0, 136, 7, 224, 13, 255, 250, 36, 0, 25, 131, 1, 2, 1, 144, 2, 2, 2, 7, 3, 2, 5, 142, 0, 113, 252, 34, 70, 39, 43, 241, 1, 113, 0, 0, 0, 0, 0, 0, 2, 113, 255, 255, 248, 248, 0, 0]
[56, 79, 250, 88, 185, 29, 25, 231, 160, 33, 42, 219, 56, 253, 163, 0, 3, 0, 13, 6, 0]

I want to create a new column where this byte_array is converted to hex.

This solution works:

test_array = [0, 136, 7, 224, 13, 255, 250, 36, 0, 25, 131, 1, 2, 1, 144, 2, 2, 2, 7, 3, 2, 5, 142, 0, 113, 252, 34, 70, 39, 43, 241, 1, 113, 0, 0, 0, 0, 0, 0, 2, 113, 255, 255, 248, 248, 0, 0]

print(bytes(test_array).hex())

Expected output: 008807e00dfffa2400198301020190020202070302058e0071fc2246272bf101710000000000000271fffff8f80000

However, I can't seem to apply this to the entire pandas dataframe. The following code, gives me incorrect results.

df['hex_column'] = bytes(df['byte_array']).hex()

Appreciate any help!

akuiper · Accepted Answer · 2018-06-17 14:03:48Z

3

Not sure if there's a vectorized solution; You might just use the apply method, and convert the series element by element:

df['hex_column'] = df['byte_array'].apply(lambda x: bytes(x).hex())

df = pd.DataFrame({'byte_array': [[1,4,136], [2, 255, 3]]})

df
#    byte_array
#0  [1, 4, 136]
#1  [2, 255, 3]

df['hex_column'] = df['byte_array'].apply(lambda x: bytes(x).hex())

df
#    byte_array hex_column
#0  [1, 4, 136]     010488
#1  [2, 255, 3]     02ff03

answered Jun 17, 2018 at 14:03

akuiper

216k33 gold badges362 silver badges379 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Kvothe Over a year ago

ah, I've got an error: TypeError: string argument without an encoding. Any idea how I can include encoding?

akuiper Over a year ago

Your list might be a string. Try convert them to list first import ast then lambda x: bytes(ast.literal_eval(x)).hex()

Kvothe Over a year ago

Ah, I think it's a problem with my dataframe. It has ~500k rows and it throws the following error: TypeError: cannot convert 'float' object to bytes Your answer is correct though. I just need to find a way to get rid of the floats. When I did astype('str') it threw the earlier error.

Kvothe Over a year ago

Actually, now that I took a closer look at the data. The column actually contained nan! Damn took me quite a while to figure this out. Anyways, I'll accept your answer, since it all works now :D

Collectives™ on Stack Overflow

Pandas byte array to hex conversion

1 Answer 1

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Related