Flatty - serializer and tools for flat binary blobs.

• atlas use flatty
• nimble install flatty

API reference

This library has no dependencies other than the Nim standard library.

• Aim of flatty is to be the fastest and simplest serializer/deserializer for Nim.
• Also includes hexprint to print out binary data.
• Also includes binny a simpler replacement for StringStream (no IO effects, operates on a string)
• Also includes hashy a hash for any objects based on the flatty serializer.
• Also includes encode a way to convert to/from utf16 BE/LE and with BOM and utf32.
• Only Nim-owned values are serialized. Raw pointers, cstrings, procs, and distinct OS handles are rejected.

Flatty aims to be fast. It achieves this by:

• Not using slowish StringStream.
• Checking if objects are "flat" and just copying them.
• Not doing anything extra like versioning, type checking, etc ...
• Liberal use of {.inline.}

Benchmark below serializes and deserializes a 11125 node tree.

name ............................... min time      avg time    std dv   runs
treeform/flatty .................... 2.265 ms      2.413 ms    +/-0.114   x100
bingo/planetis-m ................... 2.632 ms      2.705 ms    +/-0.056   x100
disruptek/frosty ................... 6.280 ms      6.532 ms    +/-0.142   x100
std/marshal ....................... 85.178 ms     93.228 ms    +/-9.956   x100
treeform/jsony ..................... 9.618 ms     10.835 ms    +/-1.276   x100

treeform/flatty .................... 2.643 ms      4.285 ms    +/-0.597   x100
bingo/planetis-m binTo ............. 5.735 ms      5.969 ms    +/-0.132   x100
disruptek/frosty ................... 9.529 ms      9.959 ms    +/-0.290   x100
std/marshal ...................... 148.723 ms    155.663 ms    +/-4.763   x100
treeform/jsony .................... 12.023 ms     13.514 ms    +/-1.143   x100

Flatty supports Nim's js mode. Some features like uint64/int64 are supported badly because of Nim's limitations. Serializing of non-Nim JavaScript objects is not supported.

By default, Flatty serializes Nim's default int, uint, and container lengths using the target's native int width. A 32-bit target uses 32-bit values, and a 64-bit target uses 64-bit values.

You can force a specific width with -d:flatty32 or -d:flatty64. Use the same mode when reading and writing a blob. Fixed-width types like int32, uint32, int64, and uint64 are not affected by these modes.

Note, unlike protobuf or thirft, flatty has no versioning mechanism, if structure of your objects changes the resulting binary would be changed and could not be read back again. Because the schema is just plain Nim types, you need to make sure changing them does not impact your ability to read old flatty binary blobs.

Flatty does not do compression it self but I recommend using the excellent https://github.com/guzba/supersnappy library to compress your flatty blobs before you send them over network or write them to disk. Snappy protocol is at this sweet spot of very fast compression, not the best but decent compression ratio, and very simple code.

The flatty + supersnappy + netty were originally made to be used together. Netty is a great for UDP networking for games.