# CAN-002: canonical.bin Memory-Mapped File

| Link | Path |
|------|------|
| **Normative** | [NORMATIVE.md](../NORMATIVE.md) |
| **Verification** | [VERIFICATION.md](../VERIFICATION.md) |
| **Tracking** | [TRACKING.yaml](../TRACKING.yaml) |
| **Spec** | [SPEC.md](../../../../resources/SPEC.md) Section 6.1 |

---

## Summary

canonical.bin MUST be a dense array of 32-byte hashes where the hash at height `h` is at byte offset `h * 32`. The file uses `memmap2` for zero-copy access, providing ~10ns lookups via OS page cache versus 1-10us for RocksDB.

---

## Specification

### File Layout

```
Offset 0:     [hash at height 0: 32 bytes]
Offset 32:    [hash at height 1: 32 bytes]
Offset 64:    [hash at height 2: 32 bytes]
...
Offset h*32:  [hash at height h: 32 bytes]

Total size = (max_height + 1) * 32 bytes
```

The array is dense: every height from 0 to max_height has a 32-byte slot. There are no gaps, length prefixes, or framing bytes.

### Read Operation

```rust
pub fn read_hash(&self, height: u64) -> Result<Option<Bytes32>> {
    let offset = height as usize * 32;
    let end = offset + 32;
    let mmap = self.mmap.read();
    if end > mmap.len() {
        return Ok(None); // height beyond current file
    }
    let mut hash = [0u8; 32];
    hash.copy_from_slice(&mmap[offset..end]);
    Ok(Some(Bytes32::new(hash)))
}
```

### Write Operation

```rust
pub fn write_hash(&self, height: u64, hash: &Bytes32) -> Result<()> {
    let offset = height as usize * 32;
    let end = offset + 32;
    let mut mmap = self.mmap.write();

    // Grow file if needed
    if end > mmap.len() {
        self.grow_file(end)?;
        // Re-acquire mmap after growth
    }

    mmap[offset..end].copy_from_slice(hash.as_ref());
    Ok(())
}
```

### Truncation

```rust
pub fn truncate(&self, max_height: u64) -> Result<()> {
    let new_size = (max_height + 1) as usize * 32;
    self.file.set_len(new_size as u64)?;
    // Remap mmap to new size
    Ok(())
}
```

### Performance Characteristics

| Operation | mmap (canonical.bin) | RocksDB (CF_CANONICAL) |
|-----------|---------------------|----------------------|
| Read | ~10ns (pointer dereference) | 1-10us (key lookup) |
| Write | ~10ns + periodic msync | ~1us (memtable insert) |
| Durability | OS-managed flush | WAL + fsync |

---

## Acceptance Criteria

- [ ] canonical.bin is a flat file with no headers or framing
- [ ] File size after storing N heights is exactly `(N) * 32` bytes (N = max_height + 1)
- [ ] Reading height `h` returns the 32 bytes at offset `h * 32`
- [ ] Writing to a height beyond current file size triggers file growth
- [ ] Truncation reduces file size to `(target_height + 1) * 32`
- [ ] File is memory-mapped via `memmap2` crate
- [ ] Zero-copy reads confirmed (no intermediate buffer allocation)

---

## Implementation Notes

- File growth should be done in chunks (e.g., 1 MB at a time) to avoid frequent `set_len` + remap calls. Pre-allocate space for ~32K heights (1 MB = 32768 * 32).
- After `set_len`, the existing `MmapMut` must be dropped and recreated, or the crate's remap facility used.
- On Linux, `madvise(MADV_SEQUENTIAL)` may improve prefetch for sequential chain sync.
- The zero-byte value `[0u8; 32]` in a slot is ambiguous (could be height 0's hash or uninitialized). In practice this is safe because height 0 is always the genesis block with a non-zero hash, and the file is only consulted for heights up to the current tip.

---

## Test Plan

| Test Name | Type | Description |
|-----------|------|-------------|
| `test_file_size_matches_height` | unit | Store hashes at heights 0..99, verify file size is 3200 bytes |
| `test_read_write_roundtrip` | unit | Write a known hash at height H, read it back, verify equality |
| `test_read_beyond_file_returns_none` | unit | Read at a height beyond the file size, verify None returned |
| `test_file_growth_on_write` | unit | Write at height 0, then at height 1000, verify file grew appropriately |
| `test_truncation` | unit | Store 100 heights, truncate to 50, verify file size is 1632 bytes and height 51+ returns None |
| `test_dense_array_no_gaps` | unit | Write at heights 0, 1, 2 (not sparse), read all three and verify correctness |

---

## Expected Test Files

- `tests/can_002_tests.rs`
