# skidbladnir `ZnippyArchive` → real `znippy-core` (one canonical archive format)

> skidbladnir hand-rolled a **JSON-indexed** archive (`src/archive.rs:354–765`,
> magic `ZNIPGAP1`) that imitates znippy and whose doc **falsely claims** it mirrors
> znippy's Arrow-IPC layout. Replace it with the REAL format via a new lean
> `znippy-core` crate, so bundles are znippy/DuckDB/Polars-readable. `**` = not done.

## Good news: clean extraction
- skidbladnir's `ZnippyArchive` is behind the **`znippy` feature (OFF by default)** — never shipped, **zero back-compat debt**. Default stays `TarZstd`.
- The real format's index code (`znippy/znippy-common/src/index.rs`) is **already lean**: deps = arrow 58 + blake3 + fst + serde — **no rayon, no znippy-cli, no compress engine**. So a `znippy-core` extraction is additive, not a churn.

## The real format (the truth)
`[blobs…][Arrow-IPC index][8-byte LE index_offset footer]` (v0.6); v0.7 adds sub-indexes + a manifest + `ZNPYMIDX` magic + reserved modules (lookup/trie/sign). Index schema (index.rs:39–70): `relative_path·chunk_seq·fdata_offset·compressed·uncompressed_size·blob_offset·blob_size·checksum(BLAKE3)`. `ZNIPPY_FORMAT_VERSION=3` in schema metadata; `check_format_version` already guards it.

## Plan
**Phase 1 — new `znippy-core` crate (znippy workspace, additive):** re-home the lean index/footer surface: `compose_index_schema`, `build_metadata_batch`, `build_arrow_metadata_for_config`, `check_format_version`, `ManifestEntry` + reserved-module consts, `read_footer`/`write_footer`, `BlobMeta`/`ChunkMeta`. Deps: arrow/anyhow/serde/log/once_cell only. **No compress, no CLI, no rayon.** (Real znippy-common can later re-export from it.)
**Phase 2 — skidbladnir consumes it:** add `znippy-core` dep behind the existing `znippy` feature; implement `RealZnippyArchive: Archive` that writes the REAL Arrow-IPC format (`pack`/`read_entry` O(1) seek/`delta`/`apply`), delete the hand-rolled `ZnippyArchive` (archive.rs:354–765). Callers (`machine.rs:413` ImageCache, `fetch.rs verify_entries`) see no change (Archive trait seam). Fix the false doc claims.
**Phase 3 (optional, holger):** holger's parallel `.tar.zst` engine (`agent/lib/src/operations.rs:254–506`) can gain a `transfer_to_znippy()` export (holger already deps znippy-compress) — tar.zst stays for non-nordisk interop.

## Gate (compat, not perf)
Round-trip: skidbladnir packs a tree → the **real `znippy` CLI lists/extracts it** byte-identically AND the Arrow index parses (DuckDB `SELECT * FROM <archive>`). O(1) selective read stays O(1). No back-compat to preserve (feature was off). Source: read-only scout 2026-06-28.
