# MessagePacker - a no-std msgpack implementation

[![crates.io](https://img.shields.io/crates/v/msgpacker?label=latest)](https://crates.io/crates/msgpacker)
[![Documentation](https://docs.rs/msgpacker/badge.svg)](https://docs.rs/msgpacker/)
[![License](https://img.shields.io/crates/l/msgpacker.svg)]()

The protocol specification can be found [here](https://github.com/msgpack/msgpack/blob/master/spec.md).

This crate targets simplicity and performance. It extends [bytes](https://crates.io/crates/bytes) `BufMut`, for increased interoperability.

The library is designed with production in mind, and will return auxiliary, useful runtime data such as buffer consumption bytes count.

## Example

```rust
use msgpacker::{MsgPacker, Packable as _, Unpackable as _};
use std::collections::HashMap;

// boilerplate derives - those aren't required
#[derive(Debug, PartialEq, Eq)]
// this convenience derive macro will implement `Packable` and `Unpackable`
#[derive(MsgPacker)]
pub struct City {
    name: String,

    // The traits are implemented for stdlib collections. If you have a custom map, you can use the
    // directive `#[msgpacker(map)]` so the traits will be automatically implemented through the
    // iterators of the map.
    inhabitants_per_street: HashMap<String, u64>,

    // This is also automatically implemented. The manual implementation is via `#[msgpacker(array)]`.
    zones: Vec<String>,

    // Binary serialization is specialized in MessagePack. We can use the directive here.
    #[msgpacker(binary)]
    metadata: Vec<u8>,
}

// create an instance of a city.
let city = City {
    name: "Kuala Lumpur".to_string(),
    inhabitants_per_street: HashMap::from([
        ("Street 1".to_string(), 10),
        ("Street 2".to_string(), 20),
    ]),
    zones: vec!["Zone 1".to_string(), "Zone 2".to_string()],
    metadata: b"foo".to_vec(),
};

// serialize the city into bytes
let buf = city.pack_to_vec();

// deserialize the city and assert correctness
let deserialized = City::unpack(&buf).unwrap();
assert_eq!(city, deserialized);
```

We also support borrowed deserialization:

```rust
use msgpacker::{MsgPackerBorrowed, Packable as _, UnpackableBorrowed as _};

#[derive(Debug, MsgPackerBorrowed, PartialEq, Eq)]
pub struct City<'a> {
    name: &'a str,
}

// create an instance of a city.
let city = City {
    name: "Kuala Lumpur"
};

let buf = city.pack_to_vec();

let deserialized = City::unpack(&buf).unwrap();
assert_eq!(city, deserialized);
```

## Features

- derive: Enables `MsgPacker` derive convenience macro.
- strict: Will panic if there is a protocol violation of the size of a buffer; the maximum allowed size is `u32::MAX`.
- assertions: Will panic on runtime if an invalid buffer is provided for prealloc serialization.
- std: Will implement the `Packable` and `Unpackable` for `std` collections.
- serde: Adds support for [serde](https://crates.io/crates/serde)

## Serde

Version `0.5.0` introduces [serde](https://crates.io/crates/serde) support.

```rust
use msgpacker::serde;
use serde_json::{json, Value};

let val = serde_json::json!({"foo": "bar"});
let ser = serde::to_vec(&val);
let des: Value = serde::from_slice(&ser).unwrap();

assert_eq!(val, des);
```

Serde performance is excelling for deserialization, but is slower for serialization.

For more information, refer to `Benchmarks`.

## Benchmarks

Results obtained with `AMD EPYC 7402P 24-Core Processor`.

The results have their correctness asserted for each benchmark item.

- pack alloc: creates a `Vec` for each serialization. Useful if the user doesn't know the size of his serialized instances. It's the most common case.
- pack prealloc: preallocates a `Vec` with the known serialization length, and executes the packing with a mutable, no-alloc slice.
- unpack owned: unpacks the structure to an owned type. Useful for complex collections such as `HashMap`.
- unpack borrowed: unpacks the structure as a borrowed type from the original bytes. Useful for very simple structures with mostly atomic types suck as `Vec<_>` and `String` (not supported by rmp-serde).

| instances count      | msgpacker | msgpacker serde | rmps      | zerompk   |
|----------------------|-----------|-----------------|-----------|-----------|
| pack alloc 1         | 67.629 ns | 251.75 ns       | 322.38 ns | 277.77 ns |
| pack alloc 10        | 333.36 ns | 764.16 ns       | 1.0981 µs | 799.47 ns |
| pack alloc 100       | 3.6478 µs | 4.2132 µs       | 6.9683 µs | 4.4160 µs |
| pack alloc 1000      | 66.081 µs | 67.055 µs       | 105.28 µs | 66.911 µs |
| pack prealloc 1      | 49.170 ns | 54.865 ns       | 94.714 ns | 46.801 ns |
| pack prealloc 10     | 293.30 ns | 367.28 ns       | 647.95 ns | 264.44 ns |
| pack prealloc 100    | 2.9884 µs | 3.5127 µs       | 6.2381 µs | 2.8508 µs |
| pack prealloc 1000   | 56.887 µs | 65.492 µs       | 102.09 µs | 52.176 µs |
| unpack owned 1       | 137.10 ns | 92.482 ns       | 251.77 ns | 138.14 ns |
| unpack owned 10      | 1.2414 µs | 1.0115 µs       | 2.2232 µs | 992.32 ns |
| unpack owned 100     | 11.820 µs | 9.2322 µs       | 22.659 µs | 9.9914 µs |
| unpack owned 1000    | 231.64 µs | 205.06 µs       | 333.20 µs | 221.33 µs |
| unpack borrowed 1    | 96.822 ns | 57.465 ns       | †         | 67.606 ns |
| unpack borrowed 10   | 868.06 ns | 662.06 ns       | †         | 572.57 ns |
| unpack borrowed 100  | 7.9413 µs | 5.8026 µs       | †         | 5.4400 µs |
| unpack borrowed 1000 | 127.29 µs | 101.97 µs       | †         | 102.96 µs |

To run the benchmarks:

```sh
RUSTFLAGS="-C target-cpu=native" cargo bench --profile bench
```

## Non-uniform collections

MessagePack is a language-agnostic format. Dynamically typed languages like Python, JavaScript, and Ruby naturally allow mixed-type collections — for instance, a Python list `[0, 1694166331209.0]` containing both an integer and a float is perfectly valid. When these values are serialized into MessagePack, the resulting byte stream encodes each element with its own type tag (`u64`, `f64`, etc.), producing an array whose elements have heterogeneous types.

Rust's type system does not directly support such collections: a `Vec<T>` requires a single concrete `T`. As noted in [#18](https://github.com/codx-dev/msgpacker/issues/18), the native `Packable`/`Unpackable` traits cannot deserialize these non-uniform arrays because they rely on a statically known element type at compile time.

The `serde` feature provides a workaround: deserialize the MessagePack bytes into `serde_json::Value`, which is a dynamically typed enum that can represent any JSON-compatible value. **This will incur performance overhead** compared to the native traits, since serde uses a visitor pattern that involves runtime type dispatch and heap allocations for every element.

```rust
use msgpacker::serde;
use serde_json::Value;

// MessagePack bytes encoding a 2-element array: [0_u64, 1694166331209.0_f64]
// This kind of payload is common when receiving data from Python, JS, or other
// dynamically typed languages that don't distinguish collection element types.
let bytes: &[u8] = &[146, 0, 203, 66, 120, 167, 66, 234, 244, 144, 0];

// Deserialize into a dynamic Value — works for any valid MessagePack payload
let value: Value = serde::from_slice(bytes).unwrap();
let items = value.as_array().unwrap();

// Each element retains its original type
assert!(items[0].is_u64());
assert!(items[1].is_f64());
```

If your use case involves only uniform collections (e.g. `Vec<u64>`), prefer the native `Packable`/`Unpackable` traits for zero-overhead deserialization.

# Note on Github

Although GitHub offers exceptional CI and hosting services virtually for free, its questionable approach towards user sovereignty and privacy is notable. Consequently, I chose to disengage from their infrastructure.

For more information, check [Give Up GitHub!](https://sfconservancy.org/GiveUpGitHub/)
