Structs and unpacking in Python

Question

I've been struggling to understand how unpacking works. More specifically this specific example in the documentation is confusing me the most because it won't give me the same output when I input it exactly the same. https://docs.python.org/2/library/struct.html#struct-examples

>>> from struct import *
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)
>>> calcsize('hhl')
8

8 byte size makes sense but when I do calcsize('hhl') it's returning 16. Storing pack('hhl', 1, 2, 3) as a variable and unpacking it works fine but using the hex values does not. I receive the error "unpack requires a string argument of length 16" , any idea what the problem is? Thanks

The doc notes that the example was done on a big-endian machine. Note that most (?) popular processors use little-endian byte ordering, so the output string may be different. As long as you unpack the string that was generated with pack, you'll be fine. You can also force a specific byte ordering (see the docs). — jedwards
– jedwards, Commented May 13, 2014 at 15:00
Yes this turned out to be the problem. Formatting it as big-endian gave the results I was expecting — John Smith
– John Smith, Commented May 13, 2014 at 15:03

zmo · Accepted Answer · 2014-05-13 15:24:49Z

it's because when you do:

>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'

it's not returning a string with [ \, x, 0, ...] but a bytearray containing each number given in hex:

>>> pack('hhl', 1, 2, 3)[0]
'\x00'

so, when you're doing:

>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')

you're actually trying to unpack a string like I was saying previously. Whereas if you do:

>>> s = pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', s)
(1, 2, 3)

it's working as expected.

Now as @falstru is telling, a way to convert a string with hex into a bytearray that struct can understand is to use binascii.unhexlify, which handles the endianness and the conversion. You can also do it yourself:

>>> unpack('hhl', str(bytearray([1,0,2,0,0,0,0,0,3,0,0,0,0,0,0,0])))
(1, 2, 3)

falsetru · Accepted Answer · 2014-05-13 14:56:50Z

1

If you mean the hex string like '0001000200000003', you need to unhexlify it first using binascii.unhexlify (or binascii.a2b_hex)

>>> import struct
>>> import binascii
>>> struct.unpack('>hhl', binascii.unhexlify('0001000200000003'))
(1, 2, 3)

answered May 13, 2014 at 14:56

falsetru

371k69 gold badges769 silver badges659 bronze badges

Collectives™ on Stack Overflow

Structs and unpacking in Python

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related