# OxiCode

A modern binary serialization library for Rust - the successor to bincode.

[![CI](https://github.com/cool-japan/oxicode/workflows/CI/badge.svg)](https://github.com/cool-japan/oxicode/actions)
[![Crates.io](https://img.shields.io/crates/v/oxicode.svg)](https://crates.io/crates/oxicode)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)

## About

OxiCode is a compact encoder/decoder pair that uses a binary zero-fluff encoding scheme. The size of the encoded object will be the same or smaller than the size that the object takes up in memory in a running Rust program.

This project serves as the spiritual successor to [bincode](https://github.com/bincode-org/bincode), maintaining **100% binary compatibility** while introducing modern improvements and advanced features that make it 150% better.

## Features

### Core Features (100% Bincode Compatible)

- **Compact encoding**: Efficient binary serialization with compact varint encoding
- **Fast**: Optimized for performance with zero-copy operations where possible
- **Flexible**: Support for various encoding configurations
- **Safe**: Strict no-unwrap policy, comprehensive error handling
- **Modern**: Built with latest Rust practices and 2021 edition features
- **no_std support**: Works in embedded and resource-constrained environments (with `alloc` feature)
- **Bincode compatibility**: Wire-format compatible with bincode 1.x default via `config::legacy()` (equivalent to bincode 2.0's `config::legacy()` preset)
- **BorrowDecode**: Zero-copy decoding via the `BorrowDecode` trait — decode into borrowed slices without allocation; generic `BorrowDecode<'de> for &'de [T]` supported for `u16`, `u32`, `u64`, `i16`, `i32`, `i64`, `f32`, `f64` via `BorrowableSliceElement`
- **encoded_size API**: Pre-calculate exact encoded byte length without allocating via `encoded_size` / `encoded_size_with_config`
- **Fixed-array encoding**: `encode_to_fixed_array::<N>()` — encode directly into a stack-allocated `[u8; N]`
- **Sequence API**: `encode_seq_to_vec` / `decode_iter_from_slice` for streaming multi-item buffers
- **Checksum API**: `encode_with_checksum` / `decode_with_checksum` — CRC32 integrity protection (optional feature)
- **Hex display**: `encode_to_display` / `EncodedBytes` — display encoded bytes as hex without allocating a `String`

### 150% Enhancement Features (Beyond Bincode)

- **⚡ SIMD Optimization**: Hardware-accelerated array encoding (2-4x speedup)
- **🗜️ Compression**: LZ4 (fast) and Zstd (better ratio) support
- **📦 Schema Evolution**: Version tracking and automatic migration
- **🌊 Streaming**: Chunked encoding/decoding for large datasets
- **⏱️ Async Streaming**: Non-blocking async I/O with tokio
- **✅ Validation**: Constraint-based validation middleware

See [Feature Comparison](#feature-comparison) below for detailed breakdown.

## Why OxiCode?

While bincode has served the Rust community well, OxiCode brings:

1. **100% Binary Compatibility**: Drop-in replacement with identical binary format
2. **Modern Rust practices**: Built from the ground up with Rust 2021 edition
3. **Safety first**: Strict no-unwrap policy throughout the codebase
4. **Better error handling**: More informative error messages and comprehensive error types
5. **Advanced features**: SIMD, compression, streaming, async, validation - features bincode lacks
6. **Active maintenance**: Dedicated to long-term support and evolution

## Installation

Add this to your `Cargo.toml`:

```toml
[dependencies]
oxicode = "0.2"

# With serde support (for serde::Serialize/Deserialize types)
oxicode = { version = "0.2", features = ["serde"] }

# Optional features
oxicode = { version = "0.2", features = ["simd", "compression", "async-tokio"] }
```

### Feature Flags

```toml
default = ["std", "derive"]
std = ["alloc"]                        # Standard library support
alloc = []                             # Heap allocations (for no_std + alloc)
derive = []                            # Derive macros for Encode/Decode/BorrowDecode
serde = []                             # Serde integration (optional)
simd = []                              # SIMD-accelerated array encoding
checksum = []                          # CRC32 integrity checking
compression-lz4 = []                   # LZ4 compression (pure Rust, fast)
compression-zstd = []                  # Zstd compression (pure Rust via oxiarc-zstd)
async-tokio = ["tokio"]               # Async streaming with Tokio
```

## Quick Start

```rust
use oxicode::{Encode, Decode};

#[derive(Encode, Decode, PartialEq, Debug)]
struct Point {
    x: f32,
    y: f32,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let point = Point { x: 1.0, y: 2.0 };

    // Encode to bytes
    let encoded = oxicode::encode_to_vec(&point)?;

    // Decode from bytes
    let (decoded, _): (Point, _) = oxicode::decode_from_slice(&encoded)?;

    assert_eq!(point, decoded);
    Ok(())
}
```

## Derive Macros

OxiCode provides first-class derive macro support for `Encode`, `Decode`, and `BorrowDecode`.

```rust
use oxicode::{Encode, Decode, BorrowDecode};

#[derive(Encode, Decode, BorrowDecode, Debug, PartialEq)]
struct Packet<'a> {
    id: u32,
    payload: &'a [u8],  // zero-copy via BorrowDecode
}
```

### Field Attributes

| Attribute | Description |
|-----------|-------------|
| `#[oxicode(skip)]` | Skip this field during encode/decode (uses `Default::default()` on decode) |
| `#[oxicode(default)]` | Use `Default::default()` if field is missing during decode |
| `#[oxicode(flatten)]` | Inline the fields of a nested struct |
| `#[oxicode(bytes)]` | Encode `Vec<u8>` or `&[u8]` as raw bytes without a length prefix |
| `#[oxicode(with = "module")]` | Use custom encode/decode functions from `module` |
| `#[oxicode(encode_with = "fn")]` | Use a custom encode function |
| `#[oxicode(decode_with = "fn")]` | Use a custom decode function |
| `#[oxicode(rename = "name")]` | Use a different name for this field in the encoded format |
| `#[oxicode(seq_len)]` | Mark field as carrying the sequence length |

### Container Attributes

| Attribute | Description |
|-----------|-------------|
| `#[oxicode(bound = "T: Trait")]` | Override the trait bounds on the generated impl |
| `#[oxicode(rename_all = "camelCase")]` | Rename all fields using a naming convention |
| `#[oxicode(crate = "path")]` | Specify a custom path to the oxicode crate |
| `#[oxicode(transparent)]` | Treat a single-field struct as its inner type (no wrapper) |
| `#[oxicode(tag_type = "u8")]` | Set the integer type used for enum discriminants |

### Variant Attributes

| Attribute | Description |
|-----------|-------------|
| `#[oxicode(variant = 5)]` | Assign a custom discriminant value to this variant |
| `#[oxicode(rename = "name")]` | Rename this variant in the encoded format |

## Supported Types (120+)

OxiCode provides built-in `Encode`/`Decode` implementations for 120+ types:

### Primitives & Core
`bool`, `u8`–`u128`, `i8`–`i128`, `f32`, `f64`, `usize`, `isize`, `char`, `str`, `String`

### Option & Result
`Option<T>`, `Result<T, E>`

### Collections
`Vec<T>`, `HashMap<K,V>`, `HashSet<T>`, `BTreeMap<K,V>`, `BTreeSet<T>`, `BinaryHeap<T>`, `LinkedList<T>`, `VecDeque<T>`

### Smart Pointers & Slices
`Box<T>`, `Arc<T>`, `Rc<T>`, `Box<[T]>`, `Arc<[T]>`, `Arc<str>`, `Cow<'_, T>`

### Network & Time
`IpAddr`, `Ipv4Addr`, `Ipv6Addr`, `SocketAddr`, `SocketAddrV4`, `SocketAddrV6`, `Duration`, `SystemTime`

### Core Types
`Range<T>`, `RangeInclusive<T>`, `Bound<T>`, `Cell<T>`, `RefCell<T>`, `Wrapping<T>`

### Atomic Types
`AtomicBool`, `AtomicI8`–`AtomicI64`, `AtomicU8`–`AtomicU64`, `AtomicIsize`, `AtomicUsize`

### OS & Path
`OsStr`, `OsString`, `Path`, `PathBuf`

### Miscellaneous
`Ordering`, `Infallible`, `ControlFlow<B,C>`, `NonZeroU8`–`NonZeroU128`, `NonZeroI8`–`NonZeroI128`, `ManuallyDrop<T>`, `PhantomData<T>`, tuples (up to 12 elements), arrays `[T; N]`

## API Highlights

```rust
use oxicode::{Encode, Decode};

// Basic encode/decode
let bytes: Vec<u8> = oxicode::encode_to_vec(&value)?;
let (decoded, bytes_read): (T, usize) = oxicode::decode_from_slice(&bytes)?;

// File I/O
oxicode::encode_to_file(&value, "data.bin")?;
let decoded: T = oxicode::decode_from_file("data.bin")?;

// Pre-calculate size without allocating
let size: usize = oxicode::encoded_size(&value)?;

// Encode into a fixed-size stack array
let (arr, n): ([u8; 32], usize) = oxicode::encode_to_fixed_array::<32>(&value)?;

// Sequence encoding — encode multiple items into one buffer
let bytes = oxicode::encode_seq_to_vec([item1, item2, item3].into_iter())?;
let items: Vec<T> = oxicode::decode_iter_from_slice::<T>(&bytes)?.collect::<Result<Vec<T>, _>>()?;

// Hex display without allocating a String
use oxicode::EncodedBytes;
println!("{}", EncodedBytes::new(&bytes)); // prints hex
```

## Using with Serde

OxiCode provides optional serde integration for types that implement `serde::Serialize` and `serde::Deserialize`:

```rust
use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct Person {
    name: String,
    age: u32,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let person = Person {
        name: "Alice".to_string(),
        age: 30,
    };

    // Encode using serde integration
    let encoded = oxicode::serde::encode_to_vec(&person, oxicode::config::standard())?;

    // Decode using serde integration
    let (decoded, _): (Person, _) = oxicode::serde::decode_from_slice(&encoded, oxicode::config::standard())?;

    assert_eq!(person.name, decoded.name);
    assert_eq!(person.age, decoded.age);
    Ok(())
}
```

**Enable serde feature in Cargo.toml:**
```toml
[dependencies]
oxicode = { version = "0.2", features = ["serde"] }
serde = { version = "1.0", features = ["derive"] }
```

## Configuration

OxiCode supports various encoding configurations:

```rust
use oxicode::config;

// Standard configuration (default): little-endian + varint
let cfg = config::standard();

// Legacy bincode 1.0-compatible: little-endian + fixed-int
let cfg = config::legacy();

// Custom configuration
let cfg = config::standard()
    .with_big_endian()
    .with_fixed_int_encoding()
    .with_limit::<1048576>(); // 1MB limit

// Use with encoding/decoding
let bytes = oxicode::encode_to_vec_with_config(&value, cfg)?;
let (decoded, _) = oxicode::decode_from_slice_with_config(&bytes, cfg)?;
```

## Advanced Features

### Checksum (CRC32 Integrity)

Protect data against corruption with built-in CRC32 checksums:

```rust
use oxicode::checksum::{encode_with_checksum, decode_with_checksum};

let data = MyStruct { /* ... */ };

// Encode with CRC32 checksum appended
let bytes = encode_with_checksum(&data)?;

// Decode and verify checksum automatically — returns Err if checksum does not match
let (decoded, _): (MyStruct, _) = decode_with_checksum(&bytes)?;
```

Enable with `features = ["checksum"]`.

### SIMD-Accelerated Arrays

Hardware acceleration for large array operations (2-4x speedup):

```rust
use oxicode::{Encode, Decode};

#[derive(Encode, Decode)]
struct LargeDataset {
    readings: Vec<f64>,  // SIMD-accelerated when feature enabled
}

// Enable with features = ["simd"]
// Auto-detects CPU capabilities (SSE2, AVX2, AVX-512)
```

See `examples/simd_arrays.rs` for detailed usage.

### Compression

Reduce size with LZ4 or Zstd compression:

```rust
use oxicode::compression::{CompressedEncoder, CompressedDecoder, CompressionType};

// LZ4 - fast compression
let mut encoder = CompressedEncoder::new(writer, CompressionType::Lz4)?;
value.encode(&mut encoder)?;

// Zstd - better compression ratio
let mut encoder = CompressedEncoder::new(writer, CompressionType::Zstd(10))?;
value.encode(&mut encoder)?;
```

See `examples/compression.rs` for detailed usage.

### Streaming Serialization

Process large datasets incrementally:

```rust
use oxicode::streaming::{StreamingEncoder, StreamingDecoder};

// Encode items one at a time
let mut encoder = StreamingEncoder::new(writer, config)?;
for item in large_dataset {
    encoder.write_item(&item)?;
}
encoder.finish()?;

// Decode items incrementally
let mut decoder = StreamingDecoder::new(reader, config)?;
while let Some(item) = decoder.read_item::<MyType>()? {
    process(item);
}
```

See `examples/streaming.rs` for detailed usage.

### Async Streaming

Non-blocking async I/O with tokio:

```rust
use oxicode::streaming::AsyncStreamingEncoder;

// Async encoding
let mut encoder = AsyncStreamingEncoder::new(writer, config);
for item in dataset {
    encoder.write_item(&item).await?;
}
let writer = encoder.finish().await?;
```

See `examples/async_streaming.rs` for detailed usage.

### Validation Middleware

Validate data during decoding:

```rust
use oxicode::validation::{Validator, Constraints};

// Create validator with constraints
let mut validator = Validator::new();
validator.add_constraint("name", Constraints::max_len(100));
validator.add_constraint("age", Constraints::range(Some(0), Some(120)));

// Validate decoded data
validator.validate(&person)?;
```

See `examples/validation.rs` for detailed usage.

### Schema Evolution

Version your data formats and migrate gracefully:

```rust
use oxicode::versioning::{Version, VersionedEncoder};

let version = Version::new(1, 0, 0);
let mut encoder = VersionedEncoder::new(writer, version, config)?;
value.encode(&mut encoder)?;

// Decoder automatically validates version compatibility
```

See `examples/versioning.rs` for detailed usage.

## Migration from bincode

OxiCode is 100% binary-compatible with bincode. Migration is straightforward:

```rust
// Before (bincode 2.0)
use bincode::{Encode, Decode, config};
let bytes = bincode::encode_to_vec(&value, config::standard())?;
let (decoded, _) = bincode::decode_from_slice(&bytes, config::standard())?;

// After (oxicode) — same shape; the 2-arg config form is *_with_config
use oxicode::{Encode, Decode, config};
let bytes = oxicode::encode_to_vec_with_config(&value, config::standard())?;
let (decoded, _) = oxicode::decode_from_slice_with_config(&bytes, config::standard())?;
```

**Binary data is 100% compatible** - you can mix libraries:
- Data encoded with bincode can be decoded with oxicode ✓
- Data encoded with oxicode can be decoded with bincode ✓

For detailed migration guide, see [MIGRATION.md](MIGRATION.md).

## Comparison with bincode

OxiCode is the spiritual successor to bincode. In **legacy mode** (`config::legacy()`), oxicode produces byte-for-byte identical output to the bincode 1.x default wire format (little-endian, fixed-int) — the same format targeted by bincode 2.0's `config::legacy()` preset — making it a true drop-in replacement.

### Wire Format Compatibility

| Mode | Endianness | Int Encoding | Compatible with bincode? |
|------|-----------|--------------|--------------------------|
| `config::legacy()` | Little-endian | Fixed-width | Yes — 100% identical |
| `config::standard()` | Little-endian | Varint | No (more compact) |

### Feature Delta

| Capability | bincode | oxicode |
|-----------|---------|---------|
| Supported types | ~60 | **120+** |
| `BorrowDecode` / zero-copy | No | **Yes** |
| Derive field attributes | Limited | **9 attributes** |
| Container/variant attributes | No | **Yes** |
| Checksums (CRC32) | No | **Yes** (`checksum` feature) |
| Compression | No | **Yes** (LZ4, Zstd, pure-Rust Zstd) |
| Async streaming | No | **Yes** (`async-tokio` feature) |
| Validation middleware | No | **Yes** |
| Schema versioning | No | **Yes** |
| `encoded_size` | No | **Yes** |
| `encode_to_fixed_array` | No | **Yes** |
| `encode_seq_to_vec` / `decode_iter_from_slice` | No | **Yes** |
| `no_std` | Yes | **Yes** |
| SIMD acceleration | No | **Yes** (`simd` feature) |

## Feature Comparison

| Feature | bincode | rkyv | postcard | borsh | **oxicode** |
|---------|---------|------|----------|-------|-------------|
| Binary Compatibility | ✓ | ✗ | ✗ | ✗ | ✓ |
| Zero-copy | ✗ | ✓ | ✗ | ✗ | ✓ |
| no_std | ✓ | ✓ | ✓ | ✓ | ✓ |
| SIMD Optimization | ✗ | ✗ | ✗ | ✗ | ✓ |
| Compression | ✗ | ✗ | ✗ | ✗ | ✓ |
| Async Streaming | ✗ | ✗ | ✗ | ✗ | ✓ |
| Validation | ✗ | ✗ | ✗ | ✗ | ✓ |
| Schema Evolution | ✗ | ✗ | ✗ | ✗ | ✓ |
| Varint Encoding | ✓ | ✗ | ✓ | ✗ | ✓ |

## Project Status

**Version 0.2.3 - Production Ready**

All core features and enhancements complete. See [CHANGELOG.md](CHANGELOG.md) for details.

**Statistics** (as of 2026-05-08):
- **Lines of Code**: 518,445 (Rust source lines across 1,014 files)
- **Files**: 1,014 Rust files
- **Test Coverage**: 19,970 tests passing (100% pass rate, 0 skipped)
  - 18 binary compatibility tests (100% byte-for-byte identical to bincode)
  - 19,952+ feature, integration, property-based, and stress tests
- **Type Coverage**: 120+ types with full Encode/Decode support
- **Binary Compatibility**: 100% verified through cross-library testing
- **Code Quality**: ✓ Zero unwrap(), ✓ Zero warnings, ✓ All files < 2000 lines

## Project Structure

This is a workspace with the following crates:

- `oxicode`: Main library crate
- `oxicode_derive`: Procedural macros for deriving Encode/Decode
- `oxicode_compatibility`: Compatibility tests and bincode interop

## Development Principles

OxiCode follows strict development principles:

- **No warnings policy**: All code must compile without warnings
- **No unwrap policy**: All error cases must be properly handled
- **Latest crates policy**: Use latest versions of dependencies
- **Workspace policy**: Proper workspace structure with shared dependencies
- **Refactoring policy**: Keep individual files under 2000 lines

## Performance

OxiCode is designed for performance:

- **SIMD acceleration**: 2-4x speedup for large arrays (with `simd` feature)
- **Zero-copy deserialization**: Where possible
- **Efficient varint encoding**: For integers
- **Minimal allocations**: During encoding/decoding
- **Benchmark suite**: Included in `benches/`

Run benchmarks:

```bash
cargo bench
```

## Testing

```bash
# Run all tests
cargo nextest run --all-features

# Run specific feature tests
cargo test --features simd
cargo test --features compression
cargo test --features async-tokio

# Run with no-std
cargo test --no-default-features --features alloc
```

## Examples

The `examples/` directory contains comprehensive examples:

- `basic_usage.rs` - Simple encoding/decoding
- `configuration.rs` - Configuration options
- `zero_copy.rs` - Zero-copy deserialization
- `simd_arrays.rs` - SIMD-accelerated arrays
- `compression.rs` - LZ4 and Zstd compression
- `streaming.rs` - Chunked streaming
- `async_streaming.rs` - Async tokio streaming
- `validation.rs` - Validation middleware
- `versioning.rs` - Schema evolution

Run examples:

```bash
cargo run --example basic_usage
cargo run --example simd_arrays --features simd
cargo run --example compression --features compression
cargo run --example async_streaming --features async-tokio
```

## Fuzzing

OxiCode ships with [cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz) targets in the `fuzz/` directory.

```bash
# Install cargo-fuzz
cargo install cargo-fuzz

# Run the decode-slice fuzzer
cargo fuzz run fuzz_decode_slice

# Run roundtrip fuzzer
cargo fuzz run fuzz_roundtrip
```

Targets:
- `fuzz_decode_slice` — decode arbitrary bytes as multiple types (no panics)
- `fuzz_roundtrip` — encode + decode must be identity on structured data
- `fuzz_streaming` — streaming decoder on arbitrary bytes
- `fuzz_versioned` — versioned decode on arbitrary bytes

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Sponsorship

OxiCode is developed and maintained by **COOLJAPAN OU (Team Kitasan)**.

If you find OxiCode useful, please consider sponsoring the project to support continued development of the Pure Rust ecosystem.

[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-red?logo=github)](https://github.com/sponsors/cool-japan)

**[https://github.com/sponsors/cool-japan](https://github.com/sponsors/cool-japan)**

Your sponsorship helps us:
- Maintain and improve the COOLJAPAN ecosystem
- Keep the entire ecosystem (OxiBLAS, OxiFFT, SciRS2, etc.) 100% Pure Rust
- Provide long-term support and security updates

## License

Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for details.

## Acknowledgments

This project builds upon the excellent work done by the bincode team and community. We're grateful for their contributions to the Rust ecosystem.

## Related Projects

- [SciRS2](https://github.com/cool-japan/scirs) - Scientific computing library
- [NumRS2](https://github.com/cool-japan/numrs) - Numerical computing library
- [ToRSh](https://github.com/cool-japan/torsh) - PyTorch-like tensor library
- [OxiRS](https://github.com/cool-japan/oxirs) - RDF and SPARQL library
- [QuantRS2](https://github.com/cool-japan/quantrs) - Quantum computing library
