# CAN-001: Dual-Layer Canonical Index

| Link | Path |
|------|------|
| **Normative** | [NORMATIVE.md](../NORMATIVE.md) |
| **Verification** | [VERIFICATION.md](../VERIFICATION.md) |
| **Tracking** | [TRACKING.yaml](../TRACKING.yaml) |
| **Spec** | [SPEC.md](../../../../resources/SPEC.md) Section 6.1 |

---

## Summary

The canonical chain MUST use a dual-layer index: an mmap hot path (`canonical.bin`) for O(1) lookups at ~10ns, and a CF_CANONICAL cold path in RocksDB for crash recovery. The mmap file MUST be rebuilt from CF_CANONICAL on startup if missing or corrupt.

---

## Specification

The canonical chain index is split into two layers to balance performance with durability:

### Layer 1: mmap Hot Path (canonical.bin)

- A memory-mapped file containing a dense array of 32-byte block hashes.
- Provides O(1) pointer dereference at ~10ns per lookup via the OS page cache.
- This is the primary read path for all height-based lookups during normal operation.
- Not crash-safe on its own; treated as a recoverable acceleration cache.

### Layer 2: CF_CANONICAL Cold Path (RocksDB)

- A RocksDB column family with `height (u64 BE) -> hash (32 bytes)` mappings.
- Provides crash-consistent, durable storage of the canonical chain.
- Read latency is 1-10us per lookup (orders of magnitude slower than mmap).
- Serves as the source of truth for recovery.

### Startup Recovery

On `BlockStore::open()`, if `canonical.bin` is missing, has an incorrect size (not a multiple of 32), or contains data inconsistent with CF_CANONICAL, it MUST be rebuilt:

```rust
fn rebuild_mmap_from_cf(&self) -> Result<()> {
    let cf = self.cf_canonical();
    let iter = self.db.iterator_cf(&cf, IteratorMode::Start);
    for item in iter {
        let (key, value) = item?;
        let height = u64::from_be_bytes(key.as_ref().try_into()?);
        let hash = Bytes32::try_from(value.as_ref())?;
        self.mmap_index.write_hash(height, &hash)?;
    }
    Ok(())
}
```

---

## Acceptance Criteria

- [ ] BlockStore creates both canonical.bin (mmap) and CF_CANONICAL (RocksDB) on initialization
- [ ] Height lookups succeed via the mmap path when canonical.bin is available
- [ ] Height lookups fall back to CF_CANONICAL when mmap is unavailable
- [ ] Deleting canonical.bin and reopening the store triggers automatic rebuild
- [ ] After rebuild, canonical.bin contents match CF_CANONICAL exactly
- [ ] A corrupt canonical.bin (wrong size or bad data) triggers rebuild, not a panic

---

## Implementation Notes

- The `memmap2` crate is used for the memory-mapped file. The `MmapMut` handle must be held in the `BlockStore` struct.
- File growth strategy: when extending the chain, the file must be resized before writing the new hash. Use `file.set_len()` followed by mmap remap.
- On Windows, mmap behavior differs slightly; ensure cross-platform tests cover file locking semantics.
- The rebuild scan iterates the entire CF_CANONICAL in key order (ascending height), which is efficient due to RocksDB's sorted key layout.

---

## Test Plan

| Test Name | Type | Description |
|-----------|------|-------------|
| `test_dual_layer_both_written` | unit | Store canonical blocks, verify both canonical.bin and CF_CANONICAL contain correct data |
| `test_mmap_rebuild_on_missing` | unit | Delete canonical.bin, reopen store, verify rebuild produces correct file |
| `test_mmap_rebuild_on_corrupt` | unit | Write garbage to canonical.bin, reopen store, verify rebuild corrects the file |
| `test_mmap_fallback_to_rocksdb` | unit | Simulate mmap unavailability, verify lookups succeed via CF_CANONICAL |
| `test_mmap_vs_rocksdb_consistency` | integration | Store 1000 canonical blocks, compare every entry in both layers for equality |

---

## Expected Test Files

- `tests/can_001_tests.rs`
