15 Aug 2011 aleix   » (Journeyer)

Packing and unpacking bit structures in Python

Last week, I released the first version of the BitPacket Python module which allows you to pack and unpack data like the struct and array modules, but in an object-oriented way. At work I needed an easy way to create network packets and at that time I did not know the existence of the struct and array modules, so I googled a bit and I found out the BitVector class for a memory-efficient packed representation of bit arrays, which I decided to use for my purpose.

I implemented three classes, BitField, BitStructure and BitVariableStructure (the lastest two are derived from BitField). A network packet would be represented by the BitStructure class, which at creation does not contain any field, and the idea is that any BitField subclass might be added to it.

I'll will show you the most basic example. Suppose, you need a simple network packet like the one below:

+---------------+-------------------+
|  id (1 byte)  |  address (4 byte) |
+---------------+-------------------+

You could easily create a network packet using BitStructure, like this:

>>> bs = BitStructure('mypacket')
>>> bs.append(BitField('id', BYTE_SIZE, 0x54))
>>> bs.append(BitField('address', INTEGER_SIZE, 0x10203040))

and print its contents:

>>> print bs
>>> (mypacket =
>>>   (id = 0x54)
>>>   (address = 0x10203040))

In order to unpack an incoming packet, we could use the variable created above or a new one without default values:

>>> bs = BitStructure('mypacket')
>>> bs.append(BitField('id', BYTE_SIZE))
>>> bs.append(BitField('address', INTEGER_SIZE))

In order to unpack an incoming array of bytes, we would do the following:

>>> data = array.array('B', [0x38, 0x87, 0x34, 0x21, 0x40])
>>> bs.set_stream(data)

We can then access to the packet fields by their name:

>>> print '0x%X' % bs['id']
0x38
>>> print '0x%X' % bs['address']
0x87342140

There are a lot more possibilities to pack and unpack bit field structures by using BitStructure and BitVariableStructure. You can see all of them in the module's online documentation.

Syndicated 2011-08-15 14:40:39 from aleix's blog

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!