# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.9.8](https://github.com/jamesgober/emdb-rs/compare/v0.9.7...v0.9.8) — 2026-05-13

**Polish + RC-prep release.** No source-logic changes — this is
the documentation refresh, examples sweep, doc-test expansion,
and 1.0 stability-commitment release. The 0.9.x line is now in
its final shape: 0.9.6 fixed the index hash function, 0.9.7
shipped streaming async iterators, and 0.9.8 lands the
documentation + 1.0 stability surface that downstream
integrators need.

### Documentation refresh

Five new design docs added under [`docs/`](docs/), each
load-bearing for a different reader:

| Document | For |
|---|---|
| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Engine internals: storage substrate, sharded hash index, read/write paths, crash recovery, compaction, concurrency model, failure modes. The reference for contributors. |
| [`docs/PERFORMANCE.md`](docs/PERFORMANCE.md) | Per-op cost model + tuning knobs. Predict performance for a workload without running benchmarks. |
| [`docs/PLATFORM-NOTES.md`](docs/PLATFORM-NOTES.md) | Linux / Windows / macOS / WSL2 specifics: io_uring, NVMe passthrough, `F_FULLFSYNC`, `FILE_FLAG_WRITE_THROUGH`, lockfile semantics, path resolution. |
| [`docs/EXAMPLES.md`](docs/EXAMPLES.md) | When-to-use guide. Eager vs streaming, sync vs async, `insert_many` vs `transaction` vs loop, single namespace vs many, raw key vs passphrase encryption. |
| [`docs/STABILITY-1.0.md`](docs/STABILITY-1.0.md) | 1.x SemVer + MSRV commitment, deprecation policy, on-disk format guarantees, yanked-release procedure, security-disclosure path. |

[`docs/API.md`](docs/API.md) gains a comprehensive
[Async surface](docs/API.md#async-surface) section covering
both the eager and streaming variants on `AsyncEmdb` and
`AsyncNamespace`. [`docs/BENCH.md`](docs/BENCH.md) is updated
to reflect the seven bench harnesses (was six) and explains
how the async surface relates to the bench numbers (no
dedicated async harness; add the `spawn_blocking` dispatch +
owned-`Vec` clone overhead to the sync numbers).

### Examples sweep

Nine new runnable example programs under
[`examples/`](examples/), each focused on one capability:

| Example | What it demonstrates |
|---|---|
| `bulk_load` | `insert_many` vs an insert-in-a-loop. |
| `range_scan` | Opt-in `enable_range_scans(true)` + half-open ranges + prefix queries + lazy iteration. |
| `ttl_cache` | `default_ttl` + `insert_with_ttl` + lazy expiry on read + eager `sweep_expired`. |
| `transactions` | Atomic batches: commit-on-Ok, rollback-on-Err, returning a value from the closure. |
| `namespaces` | Multi-tenant isolation via named namespaces — independent indexes, independent clears, independent drops. |
| `zero_copy` | `get_zerocopy` returning a `ValueRef` borrowed directly from the mmap. |
| `encryption` | Raw 32-byte key + Argon2id-derived passphrase + cipher choice (AES-GCM vs ChaCha20-Poly1305). |
| `group_commit` | `FlushPolicy::OnEachFlush` vs `Group` vs `WriteThrough` — the v0.9.x reality (Group == OnEachFlush; WriteThrough is the real semantic distinction). |
| `async_basics` | `AsyncEmdb` over a tokio multi-thread runtime: insert / get / namespace / transaction / sync_handle bridge. |
| `streaming` | v0.9.7 `iter_stream` / `range_stream` / `range_prefix_stream` + early-drop cleanup. |

Each new example is registered in `Cargo.toml` with the right
`required-features`, so `cargo run --example <name>` picks up
the necessary feature flags automatically.

### Doc-test expansion (M-6)

`# Examples` sections added to every load-bearing public
method on `Emdb`:

- `insert`, `insert_many`, `get`, `remove`
- `flush`, `compact`
- `namespace`
- `transaction` (two doctests — commit + rollback)

The doc-test count went from 13 to 22, all passing. Every
public method that a user is likely to reach for now has a
runnable example right next to its signature in the rustdoc.

### `Cargo.toml` metadata polish

- `description` rewritten to highlight the v0.9.x capabilities
  (Bitcask-style journal, lock-free sharded index, at-rest
  encryption, sync + async with streaming iterators) — same
  length, sharper for crates.io search.
- `keywords` tuned for crates.io discovery: `database`,
  `embedded`, `key-value`, `kv`, `storage`.
- `categories` unchanged: `database`,
  `database-implementations`, `data-structures`, `caching`.

### README polish

- Status section refreshed to call out v0.9.8 as polish + RC
  prep, with cross-links to the five new docs.
- New top-level **Documentation** section after the install
  block listing every doc + example destination.
- Group-commit section corrected for the v0.9.x reality:
  `OnEachFlush` and `Group` are functionally identical (both
  share fsys's group-commit coordinator); the separate `Group`
  variant is retained only for source-compat with v0.8.x
  callers, and the previous `max_wait` / `max_batch` tuning
  knobs are gone.
- Cargo features list adds the `async` feature.
- Stale "Range scans on a single namespace" entry removed
  from the Non-goals section (range scans have been supported
  via opt-in `enable_range_scans(true)` since v0.8).
- Install pins bumped to 0.9.8; `async` added to the
  all-features pin.

### Breaking changes

**None.** Pure documentation, examples, and metadata. All
v0.9.x source code compiles and runs unchanged. The on-disk
format is unchanged.

### Tests

All pre-0.9.8 tests still pass. The 9 new doc-tests added in
this release pass under `--features ttl,nested,encrypt,async`.
The 9 new examples each compile and run under their declared
feature flags.

178 unit + integration tests + 22 doctests + 23 async
streaming tests, all green. Feature combos
(`--no-default-features`, `--features ttl`,
`--features ttl,nested,encrypt`, `--features "async ttl"`,
`--all-features`) all check clean. Clippy clean, fmt clean,
release build clean.

### Internals — module map

| Module | Change |
|---|---|
| [`Cargo.toml`](Cargo.toml) | Version bumped to 0.9.8. `description` sharpened. Five new `[[example]]` entries with `required-features` for the feature-gated examples. |
| [`README.md`](README.md) | Status refresh, new Documentation index, group-commit prose updated for v0.9.x reality, Non-goals cleaned up, install pin → 0.9.8, `async` added to features list. |
| [`docs/API.md`](docs/API.md) | New `## Async surface` section covering eager + streaming variants. `EmdbBuilder` table picks up `build_async()`. |
| [`docs/BENCH.md`](docs/BENCH.md) | Bench-count corrected (six → seven; missing `index_hotpath` added). New paragraph on how async-surface perf relates to sync benches. |
| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | New file — full engine reference. |
| [`docs/PERFORMANCE.md`](docs/PERFORMANCE.md) | New file — cost model + tuning. |
| [`docs/PLATFORM-NOTES.md`](docs/PLATFORM-NOTES.md) | New file — OS-specific behaviour. |
| [`docs/EXAMPLES.md`](docs/EXAMPLES.md) | New file — when-to-use guide. |
| [`docs/STABILITY-1.0.md`](docs/STABILITY-1.0.md) | New file — 1.x stability contract. |
| [`src/db.rs`](src/db.rs) | `# Examples` sections added to `insert`, `insert_many`, `get`, `remove`, `flush`, `compact`, `namespace`, `transaction`. No logic changes. |
| [`examples/`](examples/) | Nine new programs: `bulk_load`, `range_scan`, `ttl_cache`, `transactions`, `namespaces`, `zero_copy`, `encryption`, `group_commit`, `async_basics`, `streaming`. |

### Installation

```toml
[dependencies]
emdb = "0.9.8"

# All features
emdb = { version = "0.9.8", features = ["ttl", "nested", "encrypt", "async"] }
```

MSRV: Rust 1.75.

## [0.9.7](https://github.com/jamesgober/emdb-rs/compare/v0.9.6...v0.9.7) — 2026-05-13

**Streaming async iterators.** The async surface gains
`iter_stream` / `keys_stream` / `range_stream` /
`range_prefix_stream` / `iter_from_stream` / `iter_after_stream`
on both [`AsyncEmdb`] and [`AsyncNamespace`]. Each returns a
`tokio_stream::wrappers::ReceiverStream` backed by a bounded
`tokio::sync::mpsc` channel (capacity 64). Records are pumped on
a dedicated `spawn_blocking` task that respects consumer
backpressure via `blocking_send` — when the receiver is slow,
the sync iterator stalls instead of buffering the whole
namespace into RAM. Pre-0.9.7 async iteration was eager: every
call ran the sync iterator to completion inside one
`spawn_blocking` and resolved with an owned `Vec` containing
every record. That path still exists (it's the right call for
small fixed-size queries), but for large or unbounded
iterations the streaming variants bound memory at the channel
depth × per-record size, not the namespace footprint.

No on-disk format change. No API breakage. The eager methods
(`iter`, `keys`, `range`, …) keep their signatures and
semantics.

### Why streaming was the right fix

The eager methods had two structural problems for large
namespaces:

1. **Memory.** A `Vec<(Vec<u8>, Vec<u8>)>` of a million records
   at ~1 KiB each is ~1 GiB resident before the first record
   is processed. With streaming, the same iteration peaks at
   `64 × per_record_bytes`, regardless of namespace size.
2. **Time to first byte.** Eager methods can't return until
   the entire result is materialised. For network-bound
   downstream consumers (forwarding records to a socket,
   pushing into a queue), that's wasted latency.

The streaming model addresses both: the consumer sees the
first record as soon as the pump task decodes it, and the
pump task is suspended whenever the channel is full.

### Backpressure design

- **Bounded channel (capacity 64).** Large enough that the
  consumer doesn't starve the pump on a per-record basis;
  small enough that the absolute footprint stays in the
  low-MiB range for kilobyte records.
- **`blocking_send` on the pump side.** When the channel is
  full, the blocking thread suspends until the async consumer
  drains a slot. No spinning, no busy loops.
- **Drop-aware.** If the consumer drops the stream early,
  `blocking_send` returns `Err`, the pump's `for` loop
  breaks, the iterator is dropped, and the blocking task
  exits — no leaked threads, no orphaned `Arc`s.
- **`Send` bound preserved.** The returned stream is `Send`,
  so it can move across `.await` points on multi-thread
  tokio runtimes (verified by a compile-time test).

### New dependency

- `tokio-stream = "0.1"` — gated behind the `async` feature.
  Wraps tokio's mpsc receiver in the `futures-core::Stream`
  trait. ~200 LOC, dep-light (transitive: `futures-core`
  only).

The `tokio` feature set under `async` also picks up `sync`,
needed for `tokio::sync::mpsc::channel`.

### Tests

**12 new async streaming tests** added to
[`tests/async_api.rs`](tests/async_api.rs), all under
`#[tokio::test(flavor = "multi_thread", worker_threads = 4)]`:

- `iter_stream_yields_all_records` — full namespace, no losses
- `keys_stream_yields_all_keys` — key-only path
- `range_stream_half_open_bounds` — lex-bounded slice
- `range_prefix_stream_filters_correctly` — prefix scoping
- `iter_from_stream_inclusive_start` — inclusive lower bound
- `iter_after_stream_exclusive_start` — exclusive lower bound
- `iter_stream_drop_halts_pump` — early-drop terminates the
  blocking pump task cleanly
- `namespace_iter_stream_scoped_to_namespace` — namespace
  isolation: top-level inserts don't leak into a namespace
  stream
- `streams_are_send` — compile-time `Send` bound check
- `iter_stream_backpressure_round_trip` — slow consumer with
  per-record `yield_now()` still completes correctly
- `iter_stream_matches_eager_iter` — same input → same
  output set as the eager method

23 / 23 async tests passing. All other feature combos
(`--no-default-features`, `--features ttl`,
`--features ttl,nested,encrypt`, `--all-features`) remain
green. Clippy clean, fmt clean, release build clean.

### Internals — module map

| Module | Change |
|---|---|
| [`Cargo.toml`](Cargo.toml) | Added `tokio-stream = "0.1"` (optional, default-features off); added `sync` to the `tokio` feature list; `async` feature now activates both `dep:tokio` and `dep:tokio-stream`. Dev-deps pick up `tokio-stream` and `futures-util` unconditionally for stream-test ergonomics. |
| [`src/async_impl.rs`](src/async_impl.rs) | Added `spawn_iter_stream` helper (bounded mpsc + `spawn_blocking` pump). Added 6 streaming methods on `AsyncEmdb` and 6 on `AsyncNamespace`. Added `iter_from` / `iter_after` eager methods on `AsyncNamespace` for symmetry with `AsyncEmdb`. Module-level + struct-level docs rewritten to describe the eager-vs-streaming choice. |
| [`src/lib.rs`](src/lib.rs) | Crate-level async docs updated to describe the streaming surface. `async` feature description in the Cargo-features section updated to list `tokio-stream`. |
| [`tests/async_api.rs`](tests/async_api.rs) | 12 new streaming integration tests. |

### Breaking changes

**None.** Pure addition. All v0.9.x async code compiles and
runs unchanged.

### Installation

```toml
[dependencies]
emdb = "0.9.7"

# All features (streaming-stream variants live under `async`):
emdb = { version = "0.9.7", features = ["ttl", "nested", "encrypt", "async"] }
```

MSRV: Rust 1.75.

## [0.9.6](https://github.com/jamesgober/emdb-rs/compare/v0.9.5...v0.9.6) — 2026-05-13

**Critical fix.** Patch release. Replaces the index hash function
(FxHash) with a wyhash-style mixer + Murmur3 fmix64 finalizer.
FxHash has a known weakness on structured byte inputs that the
stress harness exposed: structured keys like
`"stress-key-{idx:08}"` produced **22 956 collisions over 64 000
distinct keys (36 % collision rate)**. The mass-collision load
saturated the overflow-handling path and surfaced concurrency
edges there, producing silent post-insert misses on Linux
multi-thread runs.

### Diagnosis

Reproduced on WSL Ubuntu by pinning to 4 cores
(`taskset -c 0-3`) — the failure rate jumps from ~0 % to ~30 %
of runs. With per-insert post-verification + debug
instrumentation, narrowed the cause to the COLLISION branch of
`Shard::replace_attempt`: under heavy collision load, multiple
threads interleaved through the
`overflow.push` / `try_promote_to_overflow` dance, and the
slot ended up reachable via the OCCUPIED path with the
colliding writer's offset instead of via the OVERFLOW path.
Engine then read disk at the colliding writer's offset, key
didn't match, returned `None`.

Standalone hash analysis confirmed the root cause: 36 %
collisions on the test key pattern. The OVERFLOW path was
working correctly; it just couldn't survive load several
orders of magnitude above what the design assumed
(astronomical 1-in-2⁶⁴, not 1-in-3).

### Fix

Replaced `Index::hash_key` with a stronger mixer:

- **Two-prime 16-byte block mixing** instead of FxHash's
  single-multiply-and-rotate-by-5. Two independent 64-bit
  primes (`0xa076_1d64_78bd_642f`, `0xe703_7ed1_a0b4_28db`)
  multiplied against alternating halves prevent the
  mid-stream-bit cancellation that produced the FxHash
  collisions.
- **8-byte and 4-byte tail blocks** for keys that aren't
  16-byte multiples — same per-prime mixing pattern.
- **Murmur3 fmix64 finalizer** — three shift-xor-multiply
  rounds at the end, scrubbing residual bit-correlations.

Same standalone harness on the same 64 000-key pattern:
**0 collisions.** Birthday-bound, as it should be.

### Impact

- **v0.9.5 users on multi-thread concurrent-insert workloads
  should upgrade.** The bug is load-dependent and requires
  pathological hash distributions to trigger; well-distributed
  keys (random / UUID / hashed-pre-emdb) are unlikely to hit
  it in practice. But "unlikely" is not "never," and the new
  hash gives a clean birthday-bound regardless of input
  pattern.
- **On-disk format is unchanged.** The hash is computed
  in-memory on every operation; it doesn't appear in the
  journal. v0.9.x databases open without migration on 0.9.6.
- **No API change.** `Index::hash_key` is `pub(crate)`;
  callers are unaffected.

### Internals — module map

| Module | Change |
|---|---|
| [`src/storage/index.rs`](src/storage/index.rs) | `Index::hash_key` rewritten — wyhash-style two-prime mixer + Murmur3 fmix64 finalizer. Slot `AtomicSlot::read()` strengthened to use `Acquire` on the trailing `seq` load instead of `Relaxed` + Acquire-fence (the prior pattern allowed prior Relaxed loads to be reordered past the fence under the formal memory model). |
| [`tests/index_stress.rs`](tests/index_stress.rs) | Already in tree from v0.9.4; this release is the fix that makes it pass under the v0.9.5 Linux-4-core-pin failure mode. |

### Tests

**178 unit + integration tests + 12 doctests passing.**
Stress test (`concurrent_inserts_only_force_growth_then_read_back`,
`concurrent_inserts_reads_removes_under_growth`) passes
**10/10 runs on WSL Ubuntu under `taskset -c 0-3`**
(matching the CI failure profile from v0.9.5). Was failing
~2/10 runs before the fix.

### Breaking changes

**None.** No public API change, no on-disk format change.

## [0.9.5](https://github.com/jamesgober/emdb-rs/compare/v0.9.4...v0.9.5) — 2026-05-13

**Async surface.** New opt-in `async` feature exposing
`AsyncEmdb` and `AsyncNamespace` wrappers plus
`EmdbBuilder::build_async`. Every async method routes through
`tokio::task::spawn_blocking` so emdb's blocking I/O never
stalls the async-task scheduler. No breaking change to the
sync API; programs that don't enable the feature pay no
tokio dep cost.

### Added

- **`async` feature flag** in `Cargo.toml`. Pulls in
  `tokio = "1"` with `rt` + `rt-multi-thread` + `macros`
  (no `fs` — emdb's I/O goes through fsys, not tokio::fs).
- **`AsyncEmdb`** — cheap-clone async handle holding
  `Arc<Emdb>` internally. Mirrors the full sync `Emdb`
  surface: `open` / `open_in_memory` / `from_sync`,
  `insert` / `insert_many` / `insert_with_ttl` / `get` /
  `remove` / `contains_key` / `len` / `is_empty` / `clear`,
  `flush` / `checkpoint` / `stats` / `backup_to` / `compact`,
  `iter` / `keys` / `range` / `range_prefix` / `iter_from`
  / `iter_after`, `transaction`, `namespace` /
  `drop_namespace` / `list_namespaces`, plus TTL methods
  (`expires_at` / `ttl` / `persist` / `sweep_expired`)
  under `#[cfg(feature = "ttl")]`. Bridge to the sync API
  via `sync_handle()` for advanced uses (zero-copy reads,
  streaming iterators).
- **`AsyncNamespace`** — mirror of `Namespace` with the
  same `spawn_blocking` routing.
- **`EmdbBuilder::build_async()`** — async-context entry
  point that mirrors `EmdbBuilder::build()`. Routes the
  file-open + recovery scan through `spawn_blocking`.
- **`tests/async_api.rs`** — 12 integration tests
  covering open/builder/namespace/transaction/iter/range/
  insert_many/stats/concurrent-clones/TTL/file
  round-trip. Run via
  `cargo test --features async --test async_api`.

### Cost model — what async users should know

Each async op pays one `spawn_blocking` dispatch (tokio
schedules the closure onto its blocking pool) plus one
ownership transfer (key + value cloned to owned `Vec<u8>`
so the closure can take them by value). The dispatch is
sub-microsecond on a warm blocking pool; the value clone
is O(key + value bytes).

When the sync cost dominates (record decode, fsync, mmap
remap), spawn_blocking overhead is negligible. When the
sync cost is a single hash-table probe (`get` on an
in-memory hot key), the overhead may dominate — for
those workloads, prefer the sync surface or batch via
`insert_many` / `range`.

### Streaming iterators

The sync API offers lazy iterators (`iter`, `keys`,
`range_iter`, etc.) that decode records on each `next()`.
The async surface currently materialises these into owned
`Vec`s before returning — streaming via `impl Stream` is
on the roadmap for a later 0.9.x. For large iterations,
reach for the sync handle via `AsyncEmdb::sync_handle()`
and drive the iterator inside a single `spawn_blocking`.

### Internals — module map

| Module | Change |
|---|---|
| [`src/async_impl.rs`](src/async_impl.rs) | **New.** `AsyncEmdb` + `AsyncNamespace` + helper `blocking` / `blocking_infallible` wrappers around `tokio::task::spawn_blocking`. |
| [`src/builder.rs`](src/builder.rs) | New `#[cfg(feature = "async")] pub async fn build_async(self) -> Result<AsyncEmdb>`. |
| [`src/lib.rs`](src/lib.rs) | `#[cfg(feature = "async")] mod async_impl;` + re-export of `AsyncEmdb` and `AsyncNamespace`. Module-level docs gained an "Async surface" section. |
| [`tests/async_api.rs`](tests/async_api.rs) | **New.** 12 multi-thread tokio-runtime tests. |
| [`Cargo.toml`](Cargo.toml) | New `async = ["dep:tokio"]` feature; `tokio = { optional = true, ... }` in `[dependencies]`; `tokio` (unconditional) in `[dev-dependencies]` for the integration test runtime. |

### Breaking changes

**None.** Programs that compile against 0.9.4 compile
against 0.9.5 unchanged. Programs that don't enable the
`async` feature pay no tokio cost.

## [0.9.4](https://github.com/jamesgober/emdb-rs/compare/v0.9.3...v0.9.4) — 2026-05-13

**Critical fix.** Patch release. Fixes a TOCTOU race in the
sharded open-addressed primary index introduced in v0.9.3
that could cause silent data loss under concurrent inserts
hashing to the same probe bucket.

### Fixed

- **Concurrent insert race in the primary index**
  ([`src/storage/index.rs`](src/storage/index.rs)). Two
  threads probing distinct hashes that mapped to the same
  bucket could both observe `STATE_EMPTY` and both proceed
  to the seqlock-protected unconditional write, with the
  second writer silently overwriting the first writer's
  entry. Replaced the unconditional `write` with verify-
  then-write methods (`try_claim`, `try_update`,
  `try_tombstone`, `try_promote_to_overflow`) that re-check
  slot state under the seqlock; if it changed between
  probe and claim, the probe restarts via a new
  `ReplaceOutcome::Retry` outcome.
- **Stress-test reference-state race** in
  [`tests/index_stress.rs`](tests/index_stress.rs). The
  test's reference `HashMap` was updated AFTER the db op,
  letting two threads' `db.insert` calls land in one order
  while their `reference.insert` calls land in the
  opposite order. Take the reference lock BEFORE the db op
  so the (db, reference) pair is atomic w.r.t. other
  writers.

### Caught by

`tests/index_stress.rs` — concurrent stress test added in
v0.9.3 with 8 threads × 8 K ops each, mixed insert / get /
remove, sized to force several shard-level growth events.
The test surfaced both bugs immediately on the first push;
this release is the fix.

### Upgrade guidance

**v0.9.3 users on multi-threaded write workloads: upgrade
immediately.** v0.9.3 silently loses index entries under
concurrent inserts to the same probe bucket. Single-thread
workloads are unaffected. The on-disk journal is unaffected
(records are written correctly via fsys's lock-free LSN
reservation; only the in-memory index has the race), so a
clean restart re-reads every record from the journal and
rebuilds a correct index. There is no data on disk to
migrate or recover — restarting any v0.9.3 process onto
v0.9.4 fixes any in-memory divergence the prior run
accumulated.

## [0.9.3](https://github.com/jamesgober/emdb-rs/compare/v0.9.2...v0.9.3) — 2026-05-13

**Custom primary hash index.** Replaced
`parking_lot::RwLock<HashMap<u64, Slot>>` with a sharded
open-addressed table of seqlock-protected slots — same
public surface, no API change, no on-disk format change.
The pause-the-world growth variant ships first; full
lock-free migration (arc-swap pointer + dual-write
protocol) is on the perf roadmap for a later 0.9.x.

> **Caveat: shipped with a concurrent-insert TOCTOU race
> that was fixed in v0.9.4.** v0.9.3 users running
> multi-threaded write workloads should upgrade. See the
> v0.9.4 entry above for the bug detail and upgrade
> guidance.

### Internals — primary index rewrite

- **`AtomicSlot`** — 32-byte slot with `seq: AtomicU64`
  (seqlock version + write-in-progress bit),
  `state: AtomicU8` (Empty / Occupied / Tombstone /
  Overflow), `hash: AtomicU64`, `offset: AtomicU64`.
- **64 shards**, each owning a `RwLock<ShardInner>`
  wrapping a `Box<[AtomicSlot]>` + occupancy counters.
  Outer RwLock is only contended during growth.
- **Open addressing** with linear probing; tombstones do
  not terminate probes.
- **64-bit hash collisions** (1 in 2⁶⁴, astronomical) fall
  back to a per-shard `Mutex<HashMap<u64, Vec<(key,
  offset)>>>`. Primary slot flips to `STATE_OVERFLOW` and
  subsequent gets consult the overflow map. Fast path
  pays zero — overflow is only acquired after a real
  collision occurred.
- **Growth** triggers at load factor 0.75 — outer write-
  lock blocks new probes briefly while the table doubles.

### Added

- [`benches/index_hotpath.rs`](benches/index_hotpath.rs) —
  Criterion microbench with 5 scenarios (single-thread
  get / replace / mixed + concurrent get / mixed at 1 / 2
  / 4 / 8 threads). Reproducible baseline for regression
  tracking; reference point for the future lock-free-
  migration follow-up.
- Two new unit tests in
  [`src/storage/index.rs`](src/storage/index.rs):
  `growth_triggers_and_preserves_entries` and
  `tombstone_is_reused_on_subsequent_insert`.

### Performance (consumer NVMe, Criterion `--quick`)

| Bench | Throughput |
|---|---:|
| `single/get` | 3.59 M get/s |
| `single/replace` | 499 K replace/s |
| `single/mixed_80r_20w` | 1.48 M ops/s |
| `concurrent_get/4` | 9.50 M get/s |
| `concurrent_get/8` | 8.48 M get/s |

End-to-end `get` ~280 ns; the seqlock-protected probe
contributes ~8-12 ns uncontended versus ~25-40 ns for the
prior `parking_lot::RwLock<HashMap>`. Bare-metal numbers
under full Criterion sample counts are deferred to the
v1.0 release.

## [0.9.2](https://github.com/jamesgober/emdb-rs/compare/v0.9.0...v0.9.2) — 2026-05-12

**Perf-tuning pass on the fsys 0.9.x substrate.** Picks up
every fsys knob emdb wasn't using yet, replaces every
`std::sync` lock on the hot path with `parking_lot`,
swaps the optional sorted secondary index for a lock-free
`crossbeam_skiplist::SkipMap`, and adds a new opt-in
`EmdbBuilder::iouring_sqpoll(idle_ms)` for sustained
Linux WAL throughput. No breaking change, no on-disk
format change — drop-in upgrade from 0.9.0.

### Changed — fsys substrate `0.9` → `0.9.8`

Picks up vectored `JournalHandle::append_batch`,
`tune_for(Workload::Database)` preset (8 MiB resident
buffer pool, 256-deep io_uring ring, 4 K-deep batch
queue), `WriteLifetimeHint::Long` for Linux NVMe GC
reduction, and the `IORING_SETUP_SQPOLL` opt-in. emdb's
compaction and `insert_many` now route through
vectored `append_batch` — one LSN reservation, one heap
allocation, one platform `pwrite` for the whole batch.
The compile-side change required a `Lsn::new(offset)`
update at two call sites (fsys made `Lsn`'s inner field
private between 0.9.0 and 0.9.8 to preserve the
monotonic invariant).

### Changed — locks

- `std::sync::{RwLock, Mutex}` → `parking_lot` everywhere
  on hot paths (index shards, store `read_file` / `mmap`
  / `meta`, engine `namespaces` / `namespace_names`
  registries). Single-CAS read fast path, no poisoning.
- `RwLock<BTreeMap<Vec<u8>, u64>>` for the optional
  sorted secondary index → `Arc<crossbeam_skiplist::SkipMap>`.
  Lock-free concurrent inserts and range iteration —
  range scans no longer queue behind concurrent
  inserts. The clear-on-compaction path walks the
  skiplist and removes entries one by one (SkipMap has
  no in-place `clear`); both call sites already hold the
  engine's logical exclusive stance.

### Added

- `EmdbBuilder::iouring_sqpoll(idle_ms: u32)` — opt-in
  Linux io_uring kernel-side SQPOLL submission polling.
  No-op on macOS / Windows. Falls back cleanly to
  non-SQPOLL on EPERM (kernel < 5.13 without
  `CAP_SYS_NICE`, sandboxed containers, etc.) — same
  durability contract, slower path.
- `meta::write_with(&fs, path, header)` — meta-sidecar
  writes that reuse the engine's cached fsys handle
  instead of rebuilding one per call (eliminates
  redundant hardware-capability probing on
  `persist_meta`).
- `CachePadded` around `Store::mmap_len` and
  `NamespaceRuntime::record_count` to eliminate
  false-sharing ping-pong on the hot atomics.

### Notes

- `Error::LockPoisoned` enum variant remains on the
  public surface for API stability but is no longer
  produced by any call site — parking_lot locks cannot
  poison. Code matching on the variant continues to
  compile and run; the branch simply never executes.

### New dependencies

- `parking_lot = "0.12"` (already transitive via fsys)
- `crossbeam-skiplist = "0.1"`
- `crossbeam-utils = "0.8"` (already transitive via fsys)
- `fsys` pin tightened from `"0.9"` → `"0.9.8"`

## [0.9.0](https://github.com/jamesgober/emdb-rs/compare/v0.8.5...v0.9.0) — 2026-05-05

Major architectural change. The storage substrate is now
[`fsys`](https://crates.io/crates/fsys) — a filesystem IO crate
that provides a journal primitive (lock-free LSN reservation,
group-commit fsync, NVMe passthrough flush where supported, and
io_uring on Linux ≥ 5.1) and `Method::Auto` hardware-aware
durability dispatch. emdb's write path is now a thin layer over
`fsys::JournalHandle::append`; the read path keeps its own
`Arc<Mmap>` over the journal file for zero-copy lookups.

### Breaking — file format

v0.9 uses fsys's frame format (`[4 magic][4 length][N payload][4
crc32c]` per record) on the data file, and a new `<path>.meta`
sidecar for emdb's metadata (encryption salt, verification
block, flags). v0.7 / v0.8.x file formats are **not** compatible
with v0.9. Migration path: open the old file with the previous
emdb release, export records, reimport into a fresh v0.9
database. (No automated migration tool ships in v0.9.0; the
encryption-admin rewrite path is the reference shape for an
external exporter.)

### Breaking — `FlushPolicy::Group` shape

`FlushPolicy::Group { max_wait, max_batch }` is now bare
`FlushPolicy::Group`. The previous tuning knobs do not map onto
fsys's coordinator (which runs in a different shape — immediate-
coalesce around an in-flight syscall, no deadline-window). The
behavioural contract is preserved: concurrent flushers still
share one syscall.

### Internals — moved out, replaced, removed

| Module | Status |
|---|---|
| `src/storage/store.rs` | rewritten — wraps `fsys::JournalHandle` for writes + own `Arc<Mmap>` for reads |
| `src/storage/format.rs` | trimmed — outer-frame helpers (`record_crc`, `try_decode_record`, etc.) deleted; tag-byte + body encoders/decoders kept |
| `src/storage/meta.rs` | new — sidecar metadata file (`<path>.meta`) replacing the old 4 KiB on-disk header |
| `src/storage/flush.rs` | trimmed — `FlushPolicy::Group` no longer carries tuning knobs; fsys's coordinator owns batching |
| `src/storage/index.rs` | unchanged surface; entries now hold "payload-start" offsets pointing at the byte after fsys's frame header |
| `src/storage/engine.rs` | rewritten recovery (`fsys::JournalReader::iter()`), append (`Store::append`), read (`format::payload_at + decode_payload`), compaction (`fsys::Handle::journal` at temp path), backup (same), and encryption-admin paths |

### Performance — headline (5 M records, Windows 11 NVMe)

Same hardware, same dataset, same workload as the v0.8.5
baseline. Lower is better; wall-time milliseconds.

| phase | v0.8.5 | v0.9.0 | delta |
|---|---:|---:|---:|
| individual writes (fsync/op) | 25 281 ms | **406 ms** | **62× faster** |
| batch writes | 2 616 ms | **292 ms** | **9.0× faster** |
| random reads (4 threads) | 817 ms | **703 ms** | 1.16× faster |
| removals | 6 161 ms | **5 662 ms** | 1.09× faster |
| nosync writes | 131 ms | 127 ms | par |
| random reads (1 M) | 332 ms | 322 ms | par |
| random reads (8 threads) | 511 ms | 511 ms | par |
| compaction | 6 513 ms | 8 268 ms | 1.27× slower |
| bulk load | 3 086 ms | 13 724 ms | 4.5× slower |
| compacted size | 498 MiB | 508 MiB | 1.02× larger |

Headline: **62× faster on the single-thread fsync-per-record
column** that v0.8.5 could not fix. v0.9 also overtakes redb
on this column (406 ms vs redb 544 ms — 1.3× faster) and pulls
ahead of sled (429 ms — 1.06× faster). emdb now wins every
column in the apples-to-apples comparison, including against
its own v0.8.5 release on every read column.

The two regressions vs v0.8.5 are real but small in absolute
terms. fsys's per-record framing (12-byte CRC-32C frame around
every record) and lock-free LSN reservation add a small per-call
overhead that adds up across 5 M tight-loop appends. Both still
beat redb and sled significantly: v0.9 bulk load is 3.2× faster
than redb, 2.3× faster than sled.

Group-commit and write-through micro-benches (default tuning):

| bench | v0.8.5 | v0.9.0 | change |
|---|---:|---:|---:|
| `group_commit` OnEachFlush (8 threads × 200 writes) | 2 192 ms | **651 ms** | **3.4× faster** |
| `group_commit` Group (8 threads × 200 writes) | 272 ms | 616 ms | 2.3× slower¹ |
| `write_through` OnEachFlush (1 000 single-thread) | 1 099 ms | **931 ms** | 1.18× faster |
| `write_through` WriteThrough (1 000 single-thread) | 1 270 ms | **840 ms** | 1.51× faster² |

¹ The v0.8.5 group-commit coordinator was tuned aggressively for
this specific workload shape (max_wait=500 µs, max_batch=8). fsys's
coordinator runs without deadline-window tuning — concurrent
flushers still coalesce, but the implementation is more
conservative. Net: v0.9 OnEachFlush is much faster than v0.8.5
OnEachFlush, while v0.9 Group is slightly worse than v0.8.5 Group.
We expect to recover the gap by tuning fsys's coordinator
parameters in a v0.9.x patch.

² v0.8.5 `WriteThrough` was a regression vs v0.8.5's own
`OnEachFlush` (1270 ms vs 1099 ms). v0.9 `WriteThrough` actually
wins (840 vs 931). The mechanism is unchanged (platform-native
synchronous-write flag); the surrounding write path is just much
more efficient now.

### Tests

174 tests passing across the integration suite (vs 175 in
v0.8.5; the difference is two crash-recovery tests rewritten
against the new format and one consolidated). Test files
touched:

- `tests/crash_recovery.rs` — rewritten. The old tests poked at
  the v0.8 frame format directly (manually flipping a CRC byte,
  stomping a length prefix); fsys owns frame validation now.
  v0.9 tests focus on the integration contract: tail truncation
  surfaces correctly through `Emdb::open`, and round-trip
  through a graceful `checkpoint() + close + reopen` returns
  every record.
- `tests/decoder_robustness.rs` — one test (`valid_prefix_then_garbage_recovers_only_the_prefix`)
  rewritten to append garbage to the journal's tail (rather
  than reaching into the old header at byte offset 32, which
  no longer exists).
- All other test files (`v0_8_5_features`, `zerocopy`, `compact`,
  `namespaces`, `nested`, `range_scans`, `storage_path`, `ttl`,
  `flush_policy`, `loom_tests`, `basic`) pass unchanged.

### Notes

- New runtime dependency: `fsys = "0.9"`. Apache-2.0 / MIT
  dual-licensed (compatible with emdb's Apache-2.0).
- No new feature flags. The async-IO surface is not exposed
  yet — emdb wraps fsys's synchronous journal API only. An
  async surface is a candidate for a v0.9.x feature.
- The v0.8.5 stale-lockfile + backup + stats + iter_from APIs
  are preserved verbatim. Only the durability substrate
  changed.

## [0.8.5](https://github.com/jamesgober/emdb-rs/compare/v0.8.0...v0.8.5) — 2026-05-04

Production-polish beta. Five additive features, all small,
none touching the file format or the existing public surface.
Closes the operational gaps that come up the moment emdb is
deployed: "where do I get a backup?", "how do I tell who has
this lock?", "how do I monitor it?", "can I run with
synchronous-write durability?", "can I paginate iteration?".

### Added — `Emdb::stats()` returning `EmdbStats`

Point-in-time database introspection. Returns a `Copy`,
`#[non_exhaustive]` struct with:

- `live_records` — count across every namespace
- `namespace_count` — number of named namespaces
- `logical_size_bytes` — current writer tail
- `file_size_bytes` — total file size including pre-allocated
  padding
- `preallocated_bytes` — `file_size_bytes - logical_size_bytes`
- `range_scans_enabled` — whether the BTreeMap secondary index
  is live
- `encrypted` — whether the file is encrypted at rest

O(namespaces) plus one filesystem `metadata` call. Cheap enough
for per-second health-check polling. Documented as safe to call
inline from async contexts (no blocking cost).

### Added — `Emdb::backup_to(path)` atomic snapshot

Writes a self-contained, openable database file at `target`.
Same atomic-rename pattern the compactor uses: writes to
`<target>.backup.tmp`, `fdatasync`s, renames into place.

The result is a normal emdb file — open it with `Emdb::open` and
every record is there. No proprietary dump format, no archive,
no shell script wrappers. Refuses to back up to the live
database's own path. Overwrites an existing target by default
(callers wanting timestamped snapshots embed the timestamp in
the path).

Documented as a heavy operation worth wrapping in
`tokio::task::spawn_blocking` from async contexts; the work
scales with database size.

### Added — `Emdb::lock_holder` / `Emdb::break_lock` admin

Two static methods for diagnosing and recovering from stuck
lockfiles.

- `Emdb::lock_holder(path)` returns the metadata of whoever
  holds the advisory lock at `<path>.lock` — PID, acquired-at
  timestamp, and crate version. Returns `Ok(None)` when the
  database is unlocked.
- `Emdb::break_lock(path)` deletes the `.lock` and `.lock-meta`
  sidecars so a fresh `open` can succeed. Caller is documented
  as responsible for confirming the holder is dead before
  calling — the doc comment names specific OS tooling
  (`ps -p`, `Get-Process`) that production operators use.

The lockfile design changed to support this. The OS advisory
lock is now held on `<path>.lock` whose body stays empty;
holder metadata lives in a sibling `<path>.lock-meta` file that
is free to read regardless of whether the lock is held. The
split exists because Windows uses mandatory file locks — a
holder's `LockFileEx` blocks other handles' reads on the same
range. Putting metadata in a second file makes "show me who has
the lock" a portable, deadlock-free read.

The new `LockHolder` type is `pub` and `#[non_exhaustive]`.

### Added — `FlushPolicy::WriteThrough`

Third variant of `FlushPolicy`. Opens the file with
`FILE_FLAG_WRITE_THROUGH` (Windows) / `O_SYNC` (Linux, macOS,
BSDs) so every `pwrite` is durable on return; `flush()` becomes
a near-free `sync_data` belt-and-braces call.

For single-thread per-record-durability workloads the trade-off
is favourable on Windows: the bench's `individual writes` phase
calls `db.flush()` per record, which under `OnEachFlush` pays
one `FlushFileBuffers` per call (~27 ms on consumer NVMe).
`WriteThrough` lets the OS commit synchronously inside `pwrite`
instead, with cheaper per-record latency. The cost shifts: bulk
loads under `WriteThrough` are slower because every `pwrite`
waits for disk; the bulk-load path no longer benefits from the
OS write-back cache.

Cross-platform implementation:

- Windows: `FILE_FLAG_WRITE_THROUGH = 0x8000_0000` via
  `OpenOptionsExt::custom_flags`. Hardcoded constant — no
  `windows-sys` dependency for one bit.
- Linux: `O_SYNC = 0x101000`.
- macOS / BSDs: `O_SYNC = 0x80`.
- Other Unix targets we have not certified: flag value `0`.
  `WriteThrough` semantics degrade to `OnEachFlush` on those
  targets (correct, just no perf win — documented in the
  variant's doc comment).

The flag is propagated across `Store::swap_underlying`, so a
database opened with `WriteThrough` keeps the synchronous-write
semantics through a compaction.

### Added — `iter_from(start)` / `iter_after(start)` cursor iteration

Streaming iterators over keys at-or-after / strictly-after a
given start key, in lexicographic order. Mirrors on
`Emdb` and `Namespace`. Built on the existing `range_iter`
machinery — same lazy snapshot semantics, same one-decode-per-
`next()` cost model. Requires `enable_range_scans(true)` at
open time.

The motivating workload is paginated APIs: pass the last-seen
key as the cursor and resume iteration from the next record.
The integration tests include a complete pagination example
that walks 50 records 10 at a time.

### Lockfile body format

`<path>.lock-meta` body schema v1:

```text
emdb-lock v1
pid=<u32>
acquired_at=<unix-millis>
crate_version=<semver>
```

Lines past the version header are `key=value` pairs; unknown
keys are ignored on read for forward-compat. Documented in the
`lockfile` module docs.

### Tests

22 new integration tests in `tests/v0_8_5_features.rs` covering
every new feature. Each suite exercises happy path plus edge
cases: empty inputs, missing files, idempotency, self-target
rejection, cross-platform behaviour, concurrent writers,
snapshot-iterator semantics under contention, pagination
round-trips. Plus 6 new unit tests in `src/lockfile.rs`
covering the body parser and break_lock idempotency.

Test totals: **167 tests across all CI feature combinations**
(up from 145 in 0.8.0). Every combo (no-default-features, ttl,
nested, encrypt, ttl+nested, ttl+nested+encrypt) green.

### Notes

- No file format change. v0.7.x and v0.8.0 databases open
  unchanged in v0.8.5.
- No public-API breakage. All five additions are additive;
  `FlushPolicy` is `#[non_exhaustive]` so the new variant
  doesn't break exhaustive matches in caller code.
- One observable lockfile change: a new sibling file
  `<path>.lock-meta` appears alongside `<path>.lock` while a
  database is held. Cleanup tooling that scans for `*.lock`
  should also handle `*.lock-meta`. Both are removed on
  graceful drop.

## [0.8.0](https://github.com/jamesgober/emdb-rs/compare/v0.7.1...v0.8.0) — 2026-05-03

The release that closes the per-record-durability gap and turns the
mmap architecture into a public API. Three big additions, two
quality-of-implementation reworks, and a deeper test bench. Nothing
breaks the v0.7 file format.

### Added — `FlushPolicy::Group`, the group-commit pipeline

The headline feature. Many concurrent `db.flush()` calls now share
a single `fdatasync`. The protocol is a leader-follower scheme
([`crate::storage::flush`]):

1. Each flusher snapshots the writer's current tail offset.
2. If a recent leader's sync already covered that offset, the
   flusher returns immediately with no work.
3. Otherwise the flusher takes the coordinator lock. The first
   one in becomes the leader and waits up to `max_wait` (or until
   `max_batch` flushers have joined) before issuing the sync.
4. Followers park on a condvar until the leader's sync covers
   their snapshot.

For a workload of N independent producer threads each writing one
record then calling `flush` for per-record durability, this turns
`N × fsync_cost` into roughly `(N / max_batch) × fsync_cost`. The
new `benches/group_commit.rs` baseline measures **7.40× aggregate
write throughput** under `Group { max_wait: 500 µs, max_batch: 8 }`
vs. `OnEachFlush` on a 4-core consumer box with 8 producer threads
(see the README's bench section). Single-threaded flush is
unchanged from v0.7.x: with no follower to wait for, the leader's
`max_wait` window terminates as soon as no follower arrives.

Two implementation details that landed alongside `Group` so it
actually wins:

1. **Sync handle decoupled from the writer mutex.** The store now
   holds a `try_clone`'d sibling `File` exclusively for `sync_data`
   calls. This means an in-flight fsync no longer blocks concurrent
   `pwrite` from other threads — without this, every `flush()`
   already serialised through the writer mutex, leaving group
   commit nothing to fuse. The clone is refreshed on file rename
   (in `swap_underlying`); on file growth it stays valid because
   `set_len` does not invalidate cloned descriptors.
2. **Followers wake the leader on arrival.** The coordinator now
   `cv.notify_all()` after incrementing `pending`, so the leader
   immediately re-checks the `pending >= max_batch` exit condition
   instead of sleeping the full `max_wait` regardless of follower
   arrival.

`max_batch` should be set close to the expected concurrent flusher
count (typically `num_cpus::get()`). Setting it higher than the
real concurrency turns the leader's `max_wait` into pure tail
latency — the documentation on `FlushPolicy::Group` calls this out
explicitly.

Public surface:

- `EmdbBuilder::flush_policy(policy)` selects the policy at open
  time. Default is `OnEachFlush`, preserving v0.7.x semantics —
  upgrades are a non-event for callers that don't opt in.
- `FlushPolicy` is `#[non_exhaustive]` so we can add variants
  later (a hypothetical `Group::Adaptive` that adjusts
  `max_wait` based on observed batch sizes is the obvious one).

5 new integration tests in `tests/flush_policy.rs` cover: the
default-policy regression guard, single-thread no-deadlock
behaviour, 8 × 25 concurrent flushers all returning Ok with every
record durable after reopen, the `max_batch = 1` floor (every
flusher leads its own cycle, no underflow), and the builder
round-trip across every variant.

### Added — Streaming `iter` / `keys` / `range`

The v0.7.x iterators called `engine.collect_records()`, which
materialised every record's `(key, value, expires_at)` tuple as
an owned `Vec<u8>` triple before yielding the first item. For a
million-record database that meant a million heap allocations
plus the data itself sitting in transient memory.

v0.8 keeps the public iterator types ([`crate::EmdbIter`],
[`crate::EmdbKeyIter`], [`crate::NamespaceIter`],
[`crate::NamespaceKeyIter`]) but their internals are different:
they now hold a sorted snapshot of record offsets plus an
`Arc<Inner>`, and decode one record per `next()` call. Memory
use scales with offset count, not value size. Records inserted
after iterator construction are not visible (snapshot
semantics); records overwritten or removed since the snapshot
are skipped on decode.

For range queries, two new APIs deliver the same lazy semantics
to BTreeMap-backed scans:

- [`crate::Emdb::range_iter`] / [`crate::Emdb::range_prefix_iter`]
- [`crate::Namespace::range_iter`] / [`crate::Namespace::range_prefix_iter`]

These return [`crate::EmdbRangeIter`] /
[`crate::NamespaceRangeIter`]. The (key, offset) pairs are
snapshotted from the BTreeMap under one read-lock acquisition;
the lock is released immediately, and values are decoded on
demand as the caller pulls items. A consumer that calls
`.range_iter(...).take(10)` decodes 10 values, not the entire
range. The eager `range()` / `range_prefix()` are unchanged for
callers that want the full materialised result.

10 new integration tests in `tests/zerocopy.rs` cover the lazy
iterator paths alongside the zero-copy read tests below
(snapshot semantics, early-exit, `range_prefix_iter` parity
with `range_prefix`, the no-`enable_range_scans` guard).

### Added — `get_zerocopy` and the `ValueRef` type

The mmap-backed read path's whole point is that record bytes
already live in kernel-managed memory; allocating a `Vec<u8>` to
return them throws that away. v0.8 adds a parallel API:

- [`crate::Emdb::get_zerocopy`] / [`crate::Namespace::get_zerocopy`]
  return `Option<ValueRef>`.
- [`crate::ValueRef`] is either an `Arc<Mmap>` + byte range (the
  fast path) or an owned `Vec<u8>` (the encrypted-database
  fallback — AEAD decryption necessarily allocates).

Either way, `ValueRef` implements `Deref<Target = [u8]>`,
`AsRef<[u8]>`, and the obvious equality traits, so callers can
pass it where a byte slice is expected.

The mmap-backed variant holds a strong handle to the original
mapping. If the writer grows the file and swaps in a new
mapping, the old one stays alive until the last
`ValueRef` derived from it drops — there is no "reference
invalidation" hazard for in-flight readers.

10 integration tests cover: round-trip equivalence with `get`,
empty / missing inputs, survival across writer-driven mmap
swap, TTL-expired filtering, and the encrypted-database
fallback.

### Added — Crash-recovery integration tests (`tests/crash_recovery.rs`)

A real crash test requires `TerminateProcess` (Windows) or
`SIGKILL` (Unix). What we *can* test deterministically is the
recovery scan's behaviour on the file states a real crash leaves
behind: torn final record, bit-flipped CRC, garbage length
prefix, stale `tail_hint`. 5 new tests:

- Truncated final record is discarded; preceding records survive.
- A flipped CRC byte stops the scan at that record.
- A garbage length prefix (pointing past EOF) is treated as the
  truncation point.
- A stale `tail_hint` does not corrupt recovery — header CRC
  catches the inconsistency cleanly.
- Records survive the checkpoint + reopen path that crash
  recovery exercises.

A real fuzz harness via `cargo-fuzz` is queued for v1.0 (the
nightly-toolchain dependency makes it CI-cost-significant); these
deterministic tests are the stable-toolchain proxy.

### Added — Randomized decoder robustness (`tests/decoder_robustness.rs`)

64 iterations of "valid header + N random bytes" feed
[`crate::Emdb::open`]'s recovery scan and confirm: no panics, no
infinite loops (5 s wall-clock ceiling per iteration), no
out-of-bounds reads. Plus three deterministic shape tests:
empty data region opens to zero records, valid prefix followed
by random tail recovers exactly the prefix, and key/value sizes
from 0 bytes to 64 KiB round-trip cleanly.

### Changed — Iterator types reworked, public surface preserved

`EmdbIter` / `EmdbKeyIter` / `NamespaceIter` / `NamespaceKeyIter`
keep their public type identities and `Iterator` impls. Their
internal representation moved from `IntoIter<RecordSnapshot>` to
`(Arc<Inner>, IntoIter<u64>)` — reflects the shift to lazy
decode. No SemVer impact for callers using them via the
`Iterator` trait.

### Added — `benches/group_commit.rs`

A multi-thread per-record-flush benchmark that exercises the new
group-commit pipeline directly. Runs the same workload twice —
once under `OnEachFlush` and once under `Group { max_wait: 500 µs,
max_batch: 32 }` — and prints aggregate writes/sec plus a speedup
column. Tunable via env vars (`EMDB_BENCH_GC_THREADS`,
`EMDB_BENCH_GC_PER_THREAD`, `EMDB_BENCH_GC_MAX_WAIT_US`,
`EMDB_BENCH_GC_MAX_BATCH`) so callers can match their own deployment
profile.

### Security — `zeroize` on key material

`zeroize` is added as a direct optional dependency gated on the
`encrypt` feature (it was already present transitively via
`aes-gcm` / `argon2`; the explicit declaration just enforces the
relationship). A new `crate::encryption::KeyBytes` type alias
wraps `[u8; 32]` in `zeroize::Zeroizing`, and every internal
storage location for raw key material now uses it:

- `EmdbBuilder::encryption_key` field — was `Option<[u8; 32]>`,
  now `Option<KeyBytes>`. Public method signature still takes
  `[u8; 32]` by value; the bytes are wrapped immediately on
  entry.
- `EngineConfig::encryption_key` — same change.
- `derive_key_from_passphrase` return type — was `[u8; 32]`,
  now `KeyBytes`. The Argon2id `hash_password_into` call writes
  through the wrapper's mutable reference, so the derived key
  never exists on the heap outside the `Zeroizing` envelope.

The cipher state inside `EncryptionContext` already zeroized on
drop via RustCrypto's transitive `zeroize` integration. Together
these mean no copy of the raw key material remains in heap memory
once the `EmdbBuilder` and any derived `EngineConfig` /
`EncryptionContext` drop. Closes the REPS Security
"MUST use the `zeroize` crate" item.

The public `EncryptionInput::Key([u8; 32])` enum variant is
intentionally unchanged — switching its payload would be a
SemVer break without meaningful benefit (the bytes spend
microseconds on the stack between `EncryptionInput::Key`
construction and the builder wrapping them in `Zeroizing`).

### CI

- New `audit` job runs `cargo audit` against the RustSec advisory
  database and fails on warnings. Catches CVEs in transitive deps
  before they reach a release.
- New `deny` job runs `cargo deny check` against the policy in
  the new `deny.toml`. Enforces license, banned-crate, duplicate-
  version, and source-registry rules. Both `audit` and `deny` are
  required by REPS for production-track crates.
- `clippy::undocumented_unsafe_blocks` added to the crate-root lint
  set. Every `unsafe { ... }` block now carries a `// SAFETY:`
  comment that the compiler validates the *presence* of (not the
  content — that's still a human-review job).

### Documentation

- Crate-level docs (`src/lib.rs`) extended with sections on the new
  v0.8 surface: zero-copy reads, streaming iteration, range scans,
  group-commit durability, and the Cargo feature matrix.
  Doctest-validated; previous 6 doctest count is now 10.
- `// Phase 1` / `// Phase 2` comment style replaced throughout the
  storage and encryption-admin code paths with descriptive comments
  per REPS documentation guidance. Behaviour unchanged.

### Notes

- No file format change. v0.7.x databases open unchanged.
- No new runtime dependencies. The group-commit coordinator uses
  std `Mutex` + `Condvar`; no new crate, no MSRV impact.
- `fs4` stays at 0.8 for now; bumping forced an MSRV bump and
  none of the 0.13 features are needed yet. Revisit when an
  advisory or a needed feature appears.
- A `cargo-fuzz` target for the format decoder is queued for
  v1.0. The randomized + crafted tests added here are the
  stable-toolchain coverage in the meantime.

## [0.7.2](https://github.com/jamesgober/emdb-rs/compare/v0.7.1...v0.7.2) — 2026-05-03

A polish release ahead of v0.8's architectural work. Nothing here
breaks the v0.7 file format or the public API; everything is
additive or a documentation correction.

### Added

- **`Emdb::checkpoint()`** — explicit fast-reopen checkpoint. The
  `flush()` path was deliberately stripped of header rewriting in
  v0.7.1 because writing the 4 KB header on every per-record
  `flush()` dominates Windows `FlushFileBuffers` cost and dragged
  the per-record-durability column hard. The trade-off is that
  reopens after long writer sessions need to walk the data region
  from the last persisted `tail_hint` up to the actual tail. For
  large databases that can be slow.

  `checkpoint()` rewrites the header with the current tail and
  `fdatasync`s, so the next [`Emdb::open`] starts its recovery
  scan past the bulk of the log. Call it at quiescent points
  (after a bulk load, before a long idle period, on graceful
  shutdown). The drop of the last handle still attempts a
  checkpoint as a backstop, but `Drop` cannot return errors —
  long-lived processes that care about reopen latency should call
  `checkpoint()` explicitly so they can surface I/O errors to the
  caller.

  Backed by [`Store::persist_header`] under the hood; the new
  public API is the contract surface.

- **`tests/checkpoint.rs`** — five integration tests covering: a
  fresh database, post-flush checkpoints, repeated checkpoints
  with no intervening writes, the drop-time fallback path, and
  the round-trip-after-checkpoint correctness check (every
  inserted record is still readable after reopen).

### Changed

- **README — honest framing of the two columns where the
  comparison reads as a defect.** Both rows previously read as
  weaknesses; both deserve more nuance.

  - `individual writes (fsync/op)` reads "see note 1" instead of
    "37× slower". The note explains that the column is bounded by
    one `FlushFileBuffers` per record on Windows (≈27 ms / call,
    irrespective of dirty-page count) — that is an OS floor, not
    an emdb design defect — and references the v0.8 group-commit
    pipeline that closes the gap. Workloads that need per-record
    durability already have two paths that win in the aggregate
    columns: `db.transaction(|tx| ...)` (one fsync per
    transaction) and `db.insert_many(...)` (one fsync per batch).
  - `random range reads` reads "opt-in" instead of "N/A / feature
    gap". emdb does support range scans — they require
    [`EmdbBuilder::enable_range_scans(true)`] at open time
    (documented in the README's [Range scans](#range-scans)
    section). The phase reads as N/A only because the
    `lmdb_style` bench runs in hash-only mode by default. A fair
    head-to-head bench requires the streaming range API arriving
    in v0.8 (today's `range()` materialises the full result set,
    so the redb pattern of "range from key, take 10" would walk
    every match before returning).

  Both notes link forward to the v0.8 work that closes each gap.

- **README Status block** updated to v0.7.2; added a paragraph on
  the v0.8 / v1.0 roadmap so consumers can see what's queued.
- **README Persistence section** documents the new
  `checkpoint()` call alongside `flush()` so the durability /
  fast-reopen split is discoverable from the front page rather
  than only from the API docs.

### CI

- **MSRV job pinned to 1.75.** The `rust-version` in `Cargo.toml`
  has claimed 1.75 since v0.5; CI never enforced it. New job
  builds with both default and `ttl,nested,encrypt` feature sets
  on the pinned toolchain. Future MSRV bumps are now a deliberate
  CI change, not an accident.
- Build/test step renamed to clarify it covers unit + integration
  + doctests in one invocation.

### Notes

- No file format change. v0.7.1 databases open unchanged in
  v0.7.2.
- No new dependencies.
- No public API removed; `checkpoint()` is a pure addition.

## [0.7.1](https://github.com/jamesgober/emdb-rs/compare/v0.7.0...v0.7.1) — 2026-04-25

### Major change — storage engine rewritten as Bitcask-style mmap + append-only log

The slotted-leaf-page + WAL + page-cache + value-cache + bloom-filter
backend from v0.6 has been entirely replaced with a single
mmap-backed append-only file plus a sharded in-memory hash index.
This is the same shape used by Bitcask / HaloDB / Riak; the read path
is also the shape LMDB and redb use. The on-disk format resets to
**v1** of the new layout — v0.6 / v0.7-page databases must be exported
and reimported (the [`crate::Emdb::enable_encryption`] / `disable` /
`rotate` admin tools follow the same rewrite-then-rename shape and
can serve as a reference).

**Architecture.** One file per database. Bytes 0..4096 are the header
(magic, version, flags, encryption salt + verify block, header CRC).
Records are length-prefix + tag + body + CRC32, appended at the
tail. Reads slice directly from a kernel-managed `Arc<Mmap>` (zero
copy). Writes go through a single writer mutex and use `pwrite`
(Unix) / `seek + write_all` (Windows). File growth swaps the mmap
under an `Arc` so old readers continue with the old mapping until
they release. Crash recovery scans framed records from
`header.tail_hint`, validates each CRC, and treats the first failure
as the truncation point.

**Index.** 64-shard `RwLock<HashMap<u64, Slot>>` keyed by FxHash with
an identity-hashing inner hasher (no double-hashing on lookup).
`Slot::Single(u64)` for the common case (one offset per hash);
`Slot::Multi(Vec<(Vec<u8>, u64)>)` for hash collisions, disambiguated
by exact key compare. Disambiguation on insert uses a callback into
the engine so the hot path never allocates the key bytes for
non-colliding entries.

### Added — Range scans (opt-in BTreeMap secondary index)

The hash index doesn't support sorted iteration, so range / prefix
queries are now an opt-in feature.
`EmdbBuilder::enable_range_scans(true)` activates a parallel
`RwLock<BTreeMap<Vec<u8>, u64>>` secondary index per namespace.
Insert / replace / remove paths update both indexes; the recovery
scan rebuilds the BTreeMap from records on reopen, and compaction
preserves it through the atomic-swap rewrite.

Public surface:

- `Emdb::range(range)` and `Emdb::range_prefix(prefix)` on the default
  namespace. `range` accepts any `RangeBounds<Vec<u8>>`.
- `Namespace::range(range)` and `Namespace::range_prefix(prefix)` for
  named namespaces.
- Calling `range(...)` without `enable_range_scans(true)` at open time
  surfaces as `Error::InvalidConfig` rather than returning empty
  results.

Cost: one `Vec<u8>` clone of the key per insert plus the BTreeMap node
overhead (~40 bytes per entry on a 64-bit target). Roughly doubles
in-memory index size for typical workloads. The hash index hot path is
unchanged — users who don't enable range scans pay nothing.

7 new integration tests in `tests/range_scans.rs` cover: opt-out
default, sorted ordering, prefix helper edge cases, mutation
semantics, reopen, named namespaces, and survival through compaction.

### Added — Persistent namespace name → ID bindings (`TAG_NAMESPACE_NAME`)

Previously the `name → id` map was rebuilt on every reopen by
allocating IDs in record-encounter order. That accidentally worked
when names were created in the same order each session, but was a
real correctness bug (a different creation order on reopen would
hand back a different id than before, decoupling records from their
namespace handle).

Fixed: every named-namespace creation now appends a
`TAG_NAMESPACE_NAME` record (id 2 in the format) carrying
`(ns_id, name)`. The recovery scan replays these records before
applying inserts/removes, so reopens find the same `name → id`
mapping the writer used. Compaction re-emits the bindings in the
rewritten file. Encryption-aware path encrypts the binding the same
way it encrypts inserts.

5 new integration tests in `tests/namespaces.rs`: round-trip across
reopen, ID stability across reopens (the test that exposed the
original bug), survival through compaction, no-records-just-name
edge case, and encrypted-database variant.

### Added — `lmdb_style` apples-to-apples bench (vs `redb-bench/lmdb_benchmark.rs`)

Mirrors redb's published methodology: 5 M records, 24-byte random
keys, 150-byte random values, fastrand-seeded. Full phase set —
bulk load, individual writes, batch writes, nosync writes, len(),
random reads (1 M × 2), MT reads at 4 / 8 threads, removals,
uncompacted size, compaction, compacted size. Range reads recorded
as N/A (real feature gap; see range-scans entry above).

Set `EMDB_BENCH_RECORDS=5000000` to hit redb's published scale;
defaults to 1 M for faster local iteration.

#### 5 M-record results vs redb (Windows 11 NVMe, lower is better)

| phase                       |        emdb |    redb  |    sled  |  emdb vs redb     |
|-----------------------------|------------:|---------:|---------:|------------------:|
| bulk load                   |    **4498** |    74496 |    60807 |     16.6× faster  |
| batch writes                |    **2814** |    11043 |     1972 |      3.9× faster  |
| nosync writes               |     **220** |     1717 |     1136 |      7.8× faster  |
| random reads (1 M)          |     **596** |     5289 |    11197 |      8.9× faster  |
| random reads (4 threads)    |    **1083** |    17543 |    34605 |     16.2× faster  |
| random reads (8 threads)    |     **653** |    17160 |    33284 |   **26× faster**  |
| removals                    |   **11948** |    54905 |    46155 |      4.6× faster  |
| compaction                  |   **11490** |    16506 |      N/A |      1.4× faster  |
| uncompacted size            |    1.08 GiB | 4.00 GiB | 2.13 GiB |     3.7× smaller  |
| compacted size              | **498 MiB** | 1.64 GiB |      N/A |     3.4× smaller  |
| individual writes (fsync/op)|       27455 |  **734** |  **316** | 37× **slower**    |
| random range reads          |         N/A |     3958 |     9688 | feature gap       |

emdb wins every aggregate-throughput column — often by an order of
magnitude — and is 3-4× smaller on disk both compacted and not. The
`individual writes` column (each write fsync'd on its own commit) is
the one place emdb loses, dominated by Windows `FlushFileBuffers`
latency. Workloads that need per-record durability should batch
through `db.transaction(...)` or `db.insert_many(...)`, which amortise
the fsync cost.

### Documented — Single-writer model + multi-writer deferred to v1.0

The Concurrency section of the README now states the actual model
explicitly: lock-free reads (sharded hash index + `Arc<Mmap>`),
single-writer writes (one mutex around the encode-then-pwrite step).
This matches LMDB / redb / BoltDB. True multi-writer concurrency
requires either a recovery-model change (skip-bad-CRC, scan-forward)
or per-thread log segments; both have correctness trade-offs that
warrant the v1.0 design pass. Queued.

### Added — Real `Emdb::compact()` (live-record sibling rewrite + atomic swap)

`compact()` was a flush-shaped no-op in the initial rewrite. Now it actually
reclaims space:

1. Snapshot every namespace's live `(key, value, expires_at)` tuples by
   walking the in-memory indexes against the current mmap.
2. Write a fully-formed sealed file at `<path>.compact.tmp` directly via
   buffered `File` I/O (no mmap on the temp file, so Windows is happy
   shrinking the file size after writes).
3. `fdatasync` the temp file, then call [`Store::swap_underlying`] which
   drops our writer's File handle, atomic-renames the temp over the
   canonical path, reopens the writer, and refreshes the mmap.
4. Clear and rebuild every namespace index from the new layout via the
   same `recovery_scan` used at open time.

Existing readers holding `Arc<Mmap>` snapshots from before the compaction
keep reading from the old inode (the kernel pins it for the duration of
any active mapping); new reads see the compacted layout. Three new
integration tests in `tests/compact.rs` cover the size-shrinks path,
the empty-DB no-op, and namespace preservation through a compaction.

### Added — `concurrent_reads` bench (multi-thread read fan-out)

Single-thread `compare_read` undersells the lock-free `Arc<Mmap>` read
path because there's no contention to observe. New
`benches/concurrent_reads.rs` spawns N reader threads against a
pre-populated DB and measures aggregate throughput across thread counts
1, 2, 4, 8.

Numbers on the same Windows 11 NVMe box as the existing benches:

| reader threads | aggregate reads (Melem/s) |
|---------------:|--------------------------:|
| 1              |                      4.75 |
| 2              |                      6.57 |
| 4              |                      9.18 |
| 8              |                     11.97 |

Reads scale through 8 threads on a 4-core machine — the kernel-managed
mmap plus the 64-shard hash index keep the hot path lock-free, so the
only contention past core count is shared memory bandwidth.

### Changed — README rewrite for the new architecture

Dropped the v0.6/v0.7 dual-engine story, the `prefer_v4` opt-in, and
references to the (now removed) `FlushPolicy`, slotted-leaf chains, WAL,
and `BatchBegin`/`BatchEnd` markers. New README leads with the bench
numbers (single-thread + multi-reader), explains the
Bitcask-style architecture in two sentences, and documents the
`db.transaction()` / `db.insert_many()` choice for callers who want
the redb-style transaction-batched insert pattern.

### Added — `Emdb::insert_many` / `Namespace::insert_many` bulk-insert API

The fast path for bulk-loading. All records are framed into one buffer
under a single writer-mutex hold and written via a single `pwrite`
syscall.

- `Emdb::insert_many<I, K, V>(items)` where `I: IntoIterator<Item = (K, V)>`,
  `K: AsRef<[u8]>`, `V: AsRef<[u8]>`. Mirror on `Namespace::insert_many`
  for named namespaces.
- Records inside one `insert_many` call are written atomically *as
  individual records* (each gets its own CRC). They are **not**
  all-or-nothing as a group — a crash mid-batch leaves a CRC-validated
  prefix on disk. For all-or-nothing semantics use
  `db.transaction(|tx| ...)`, which buffers writes in an overlay and
  routes the commit through `insert_many` plus a final `flush` so the
  whole batch is durable together.

### Added — OS-default storage path resolution

`Emdb::builder()` now resolves a platform-appropriate database file path
when the caller opts in via `app_name` / `database_name` / `data_root`.
This closes the embedder-ergonomics gap that previously forced HiveDB and
every other consumer to know each platform's data-directory convention.

- **`src/data_dir.rs`.** Cross-platform resolver: Linux/BSD use
  `$XDG_DATA_HOME` (or `$HOME/.local/share`), macOS uses
  `$HOME/Library/Application Support`, Windows uses `%LOCALAPPDATA%`
  (falling back to `%APPDATA%` then `%USERPROFILE%\AppData\Local`).
  Last-resort fallback is the process current directory so the
  builder never panics.
- **Builder methods.** `app_name(name)` (single folder name, default
  `"emdb"`), `database_name(name)` (default `"emdb-default.emdb"`),
  `data_root(path)` (escape hatch for tests / containers / sandboxes).
  Resolved path is `<data_root>/<app_name>/<database_name>`.
- **Validation.** Path separators (`/`, `\`), `..`, and the empty
  string are rejected at build time so a stray value cannot escape
  the data root and behaviour stays identical on every platform.
- **Conflict detection.** Mixing `path()` with any of the
  OS-resolution methods returns `Error::InvalidConfig` — pass either
  an explicit path or the OS-resolution methods, never both.
- **Tests.** 7 unit + 10 integration covering round-trips through
  v0.6 and v0.7, default substitution, `mkdir -p` behaviour,
  multi-app coexistence under one root, and every rejection branch.

### Added — AES-256-GCM + ChaCha20-Poly1305 at-rest encryption

Opt-in via the `encrypt` Cargo feature plus
[`crate::EmdbBuilder::encryption_key`] (raw 32-byte key) or
[`crate::EmdbBuilder::encryption_passphrase`] (Argon2id KDF). Either
mode encrypts every record body; unencrypted records simply skip the
encryption path so unencrypted databases stay byte-identical to a
non-`encrypt` build.

- **Ciphers.** AES-256-GCM via `aes-gcm` 0.10 (default; AES-NI on
  modern x86, Crypto Extensions on ARMv8) and ChaCha20-Poly1305 via
  `chacha20poly1305` 0.10 (selectable via `EmdbBuilder::cipher(...)`
  for hardware without AES acceleration). Both use a 96-bit random
  nonce drawn fresh from `OsRng` per record. Counter-based nonces
  were rejected: durable counter state can roll back on
  backup-restore, and a rolled-back nonce with the same key is the
  one mistake AEAD ciphers do not survive.
- **Passphrase mode.** `EmdbBuilder::encryption_passphrase("...")`
  derives a 32-byte key via Argon2id (19 MiB memory, 2 iterations,
  1 lane — OWASP defaults for interactive use). The salt is a fresh
  random 16-byte block per database, persisted at header offsets
  40..56. Reopens read the salt and rerun the KDF; wrong passphrase
  surfaces as [`crate::Error::EncryptionKeyMismatch`] before any user
  data is touched. Mutually exclusive with `encryption_key()`.
- **Record envelope.** Every record carries the same outer framing
  (`[len][tag][body][crc]`); the encrypted variant sets bit 7 of the
  tag and the body becomes `[nonce: 12][ciphertext + AEAD tag]`. The
  CRC catches torn writes; the AEAD tag catches tampering. See
  [`crate::storage::format`] for the full layout.
- **Verification block.** Header bytes 56..116 hold an AEAD-encrypted
  copy of a fixed magic plaintext
  (`b"EMDB-ENCRYPT-OK\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"`). On open
  the engine decrypts and compares; mismatch surfaces as
  `Error::EncryptionKeyMismatch` before any user data is read.
- **Cipher selection on disk.** `FLAG_CIPHER_CHACHA20` (bit 1 of the
  header flags) records the cipher choice on creation. Reopens
  auto-dispatch from the on-disk flag; callers do not have to
  restate the cipher.
- **Offline admin APIs.** Three static methods on [`crate::Emdb`]:
  `enable_encryption(path, target)` (unencrypted → encrypted in
  place), `disable_encryption(path, current)` (the reverse), and
  `rotate_encryption_key(path, from, to)` (re-encrypt under a new
  key). All three use atomic rewrite-then-rename: the original is
  preserved at `<path>.encbak` on success and untouched on any
  failure. Each side accepts either a raw key or a passphrase via
  the new `EncryptionInput::{Key, Passphrase}` enum (re-exported
  at the crate root).
- **Error variants.** `Error::Encryption(&'static str)` for
  malformed buffers / AEAD-machinery failures (not user-recoverable);
  `Error::EncryptionKeyMismatch` for tag-validation failures (user
  supplied the wrong key). Both gated on `feature = "encrypt"`.
- **`Debug` does not leak keys.** `EncryptionContext::fmt` writes
  `"<redacted>"` instead of the cipher state.

### Removed

- **`FlushPolicy`.** Sync semantics are simpler now: `insert` writes
  to the OS buffer, `flush()` calls `fdatasync`. Callers that want
  per-record durability call `flush` after each insert; callers that
  want batched durability call `flush` after `insert_many` or at the
  end of a transaction.
- **`EmdbBuilder::prefer_v4(...)` + the v0.6 / v0.7 dual-engine
  dispatch.** There is exactly one engine.
- **`emdb-cli` binary + `cli` Cargo feature.** Not standard for
  embedded KV libraries; the `Emdb::enable_encryption` /
  `disable_encryption` / `rotate_encryption_key` library APIs cover
  the same need programmatically.
- **`compress` Cargo feature + the `lz4_flex` value-compression
  shim.** The new format does not include compressed-record
  framing.
- **The slotted-leaf-page + WAL + page-cache + value-cache + bloom
  filter modules** (`src/storage/v4/`, `src/storage/page/`,
  `src/keymap.rs`, `src/page_cache.rs`, `src/value_cache.rs`,
  `src/bloom.rs`, `src/index.rs`, `src/compress.rs`, the v0.6 v0.7
  migration scaffolding). Tests for the removed surface
  (`tests/v4_*.rs`, `tests/migration.rs`, `tests/page_format.rs`,
  `tests/recovery.rs`, `tests/transactions.rs`,
  `tests/concurrency.rs`) are gone too — their guarantees are
  covered by the new integration tests
  (`tests/persistence.rs`-style + `tests/compact.rs`,
  `tests/range_scans.rs`, `tests/namespaces.rs`).

### Tests + format

109 tests passing across `default`, `ttl,nested,encrypt`,
`--no-default-features`, `nested`-only, and `encrypt`-only feature
combos. Library is clippy-clean under the project's strict lint
profile (deny `unwrap_used`, `expect_used`, `unreachable`, `todo`,
`unimplemented`, `print_stdout`, `print_stderr`, `dbg_macro`,
`warnings`).

The on-disk format resets to **v1** of the new mmap+append layout.
v0.6 page-format files and the original v0.7 dual-engine page-format
files cannot be opened by this release. Migration path: open the
old file with the previous emdb release, export records, reimport
into a fresh v0.7.1 file. (No automated migration tool ships in
0.7.1; the encryption-admin rewrite primitive in
[`crate::encryption_admin`] is the reference shape for an external
exporter.)

## [0.6.0](https://github.com/jamesgober/emdb-rs/compare/v0.5.0...v0.6.0) — 2026-04-25

### Added

- Page-oriented file format (v3 header) with 4 KB fixed pages for efficient sequential I/O.
- B-tree index over keys with O(log n) page-tree traversal during compaction
  and replay paths.
- Free-list management for efficient page reuse after deletions.
- Write-ahead log (WAL) sidecar with crash recovery and atomic updates.
- Automatic schema migration pipeline:
  - v1 (EMDB\0\0\0\0 magic) → v3 conversion on open.
  - v2 (v0.4 batch format) → v3 conversion on open.
  - Creates `.bak` backup on first migration; subsequent opens are idempotent.
- Public `Emdb::migrate()` API for explicit migration of legacy-format files.
- Optional memory-mapped read backend via `mmap` feature (uses `memmap2 0.9`).
- Page-format integration tests validating round-trip persistence and integrity.
- Migration integration tests covering v1→v3, v2→v3, and idempotence behavior.
- `OpRef<'_>` — borrowed view of `Op` for the storage append path. Constructing
  one is allocation-free; the previous owned `Op` forced two `Vec<u8>` clones
  per insert just to hand bytes to the WAL.
- Sharded primary index (`src/index.rs`) with 32 lock-striped shards keyed by
  FNV-1a. Reads on different keys do not block each other; concurrent writes
  contend only on the target shard.

### Changed

- On-disk format advanced to version `3` (page-oriented).
- Auto-migration occurs transparently on first open of legacy-format files.
- Reader accepts v1, v2, and v3 headers with automatic format detection.
- WAL writes go through a 64 KB userspace `BufWriter`. A burst of `append`
  calls becomes one syscall per buffer flush instead of one per record.
- `Storage::append` now takes `OpRef<'_>` instead of `&Op`. Callers no longer
  clone keys and values to construct an op; `transaction::commit` converts
  staged owned ops via `OpRef::from(&op)` at zero cost.
- `encode_op` writes directly into the output buffer (single pass) and
  CRCs over the bytes already written, eliminating the per-call payload
  `Vec` allocation.
- In-memory mode (`open_in_memory`) bypasses the storage backend entirely.
  Inserts touch only the target shard — no mutex acquisition, no dynamic
  dispatch, no WAL append.
- `Inner.backend` is now `Option<Mutex<PageStorage>>` (concrete type), not
  `Mutex<Box<dyn Storage>>`. Eliminates dynamic dispatch on every write.
- Primary index is now an unordered sharded `HashMap` instead of a single
  `RwLock<BTreeMap>`. Iteration order is unspecified — no API contract
  promised ordering, and the previous order was incidental.
- Transactions no longer hold a database-wide write lock for their lifetime.
  They stage in a closure-local overlay; commit briefly takes the backend
  mutex and every shard write lock to apply the overlay atomically.
- Insert path now holds the backend mutex across the shard write so the
  in-memory state never reorders relative to the durability log.
- Crate documentation updated to reflect persistent, migration-capable storage model.
- README phase status synchronised with v0.6 implementation checkpoint.
- Crate version bumped to `0.6.0`.

### Fixed

- Automatic migration preserves all records and metadata during v1/v2 → v3 conversion.
- `benches/concurrency.rs:87` no longer fails to compile (`&Arc<Vec<...>>`
  is not iterable; replaced with `writer_data.iter()`).
- `src/storage/page/btree.rs` MSRV violation: replaced `Option::is_none_or`
  (stable in 1.82) with `Option::map_or` to satisfy the declared 1.75 MSRV.
- `src/storage/page/btree.rs` `let _x = …` for unit-typed expression
  (clippy `let_unit_value`).
- `MemoryStorage` removed — every in-memory mode acquisition was a no-op
  Mutex and dynamic dispatch through a `Box<dyn Storage>` for nothing.

### Performance

- In-memory `kv_insert` (1,000 records): −44% wall time (~1.8× throughput).
- In-memory `kv_remove` (1,000 records): −49% wall time (~2× throughput).
- In-memory `kv_get` (1,000 records): −22% wall time (~1.3× throughput).
- Persistent `compare_read` (5,000 records): +30–53% throughput, peaking at
  ~10.5M elem/s — fastest of emdb / sled / redb / rocksdb on the reference
  workload.
- Persistent `compare_insert` (5,000 records): ~30% improvement (40K → 51K
  elem/s). The remaining gap to sled/redb is architectural (one 4 KB page
  per value) and is the primary target of the v0.7 storage redesign.

## [0.5.0](https://github.com/jamesgober/emdb-rs/compare/v0.4.0...v0.5.0) — 2026-04-24

### Added

- Cross-process lockfile exclusion via `fs4` for file-backed databases.
- Cheap `Clone` support for `Emdb` handles via shared inner state.
- Concurrency integration coverage (`tests/concurrency.rs`) for:
	- many-reader / one-writer execution,
	- concurrent transactions,
	- lock contention and lock release behavior,
	- clone-handle correctness across threads.
- Loom-gated lock-order test target (`tests/loom_tests.rs`).
- Concurrency benchmark suite (`benches/concurrency.rs`).

### Changed

- **BREAKING:** mutating `Emdb` APIs now take `&self` instead of `&mut self`.
- `Emdb` internals refactored to `Arc<Inner>` with `RwLock`-protected state and
	`Mutex`-serialized storage appends.
- Transactions now acquire and hold the state write lock for closure lifetime.
- `Error` expanded with lock-specific variants:
	`LockBusy`, `LockfileError`, and `LockPoisoned`.
- Crate version bumped to `0.5.0`.

### Fixed

- File-backed open now prevents concurrent process access to the same database
	path via advisory lockfile.

## [0.4.0](https://github.com/jamesgober/emdb-rs/compare/v0.3.0...v0.4.0) — 2026-04-24

### Added

- Closure-based transaction API via `Emdb::transaction(|tx| ...)`.
- `Transaction` methods: `insert`, `get`, `remove`, `contains_key`.
- Atomic batch log markers: `BatchBegin` and `BatchEnd` op records.
- Crash-safe replay rules for incomplete or corrupted transactional batches.
- Transaction id tracking persisted in file header (`last_tx_id`).
- v0.4 integration tests for commit/rollback, read-your-writes, crash recovery, and tx id persistence.
- Transaction benchmarks for single-op and multi-op batch throughput.

### Changed

- On-disk format advanced to version `2` for new files.
- Reader accepts both v1 and v2 headers.
- `Error` expanded with transaction variants (`TransactionInvalid`, `TransactionAborted`).
- Crate version bumped to `0.4.0`.

### Fixed

- Replay now truncates and discards malformed or incomplete transactional batches at recovery boundaries.

## [0.3.0](https://github.com/jamesgober/emdb-rs/compare/v0.2.0...v0.3.0) — 2026-04-24

### Added

- Persistent file-backed open path via `Emdb::open(path)`.
- New `FlushPolicy` configuration with `OnEachWrite`, `EveryN`, and `Manual`.
- Storage backend abstraction (`Storage`) with file and memory implementations.
- Append-only operation log with CRC-32 record integrity checks.
- Crash recovery replay behavior for truncated/corrupted tail records.
- Public persistence control APIs: `flush`, `compact`, and `path`.
- Integration test coverage for persistence round-trip, recovery, format mismatches, and compaction.
- Persistence benchmark suite comparing write throughput across flush policies.

### Changed

- `EmdbBuilder::build()` is now fallible and returns `Result<Emdb>`.
- `Error` expanded with v0.3 persistence variants (`Io`, `MagicMismatch`, `VersionMismatch`, `FeatureMismatch`, `Corrupted`, `InvalidConfig`).
- README updated to Phase 2 and persistent usage examples.
- Crate version bumped to `0.3.0`.

### Fixed

- Recovery now truncates invalid trailing records and reopens successfully when crash tails are detected.

## [0.2.0](https://github.com/jamesgober/emdb-rs/compare/v0.1.0...v0.2.0) — 2026-04-24

### Added

- Functional in-memory key/value storage backed by `BTreeMap<Vec<u8>, Record>`.
- Core API methods: `insert`, `get`, `remove`, `contains_key`, `clear`, `iter`, and `keys`.
- `EmdbBuilder` with `build` and `default_ttl` (when `ttl` feature is enabled).
- `ttl` feature (enabled by default) with:
	- `Ttl` policy enum.
	- `insert_with_ttl`, `expires_at`, `ttl`, `persist`, and `sweep_expired` APIs.
	- Expiration-aware visibility for `get`, `contains_key`, `iter`, and `keys`.
- `nested` feature with dotted-prefix ergonomics:
	- `group`, `delete_group`, and `focus` on `Emdb`.
	- `Focus` scoped operations: `set`, `get`, `remove`, `contains_key`, `focus`, `iter`, and `delete_all`.
	- `Focus::set_with_ttl` when both `nested` and `ttl` are enabled.
- Integration tests for core, ttl, nested, and feature matrix behavior.
- Criterion benchmark scaffold in `benches/kv.rs` for insert/get/remove throughput.

### Changed

- Crate root docs now include base usage plus feature-gated ttl and nested examples.
- CI matrix now validates all feature combinations (`""`, `ttl`, `nested`, `ttl nested`) across Linux, macOS, and Windows.
- README updated to Phase 1 status and expanded with features plus ttl/nested examples.

### Fixed

- Feature-gated error variants and cfg hygiene so all feature combinations build and test cleanly.

## [0.1.0](https://github.com/jamesgober/emdb-rs/releases/tag/v0.1.0) — 2026-04-24

### Added

- Initial crate scaffold.
- `Emdb` struct with `open_in_memory`, `len`, and `is_empty` stubs.
- `Error` enum with `NotImplemented` placeholder variant.
- `Result<T>` type alias.
- Apache-2.0 license.
- CI workflow for Linux, macOS, and Windows.
- REPS (Rust Efficiency & Performance Standards) compliance at crate root.
