# **Production-grade `stream_v2` project layout**

This layout is **deliberate**:

* minimal compile blast radius
* clean dependency direction
* testability without PyO3 / Tokio
* zero circular imports

---

## 📂 `src/stream_v2/` — **Final Layout**

```bash
src/stream_v2/
│
├── mod.rs                    # Public façade + re-exports
│
├── core.rs                   # encrypt_stream_v2 / decrypt_stream_v2
│                             # (stable, public API)
│
├── io.rs                     # InputSource / OutputSink
│                             # ordered writer / reader normalization
│
├── pipeline.rs               # Thread-pool wiring (no crypto)
│
├── segment_worker/           # SEGMENT = unit of parallelism
│   ├── mod.rs
│   ├── types.rs              # SegmentInput / EncryptedSegment / DecryptedSegment
│   ├── crypto.rs             # SegmentCryptoContext (keys, header, nonce policy)
│   ├── encrypt.rs            # EncryptSegmentWorker
│   ├── decrypt.rs            # DecryptSegmentWorker
│   └── tests.rs              # segment worker tests (FAST)
│
├── frame_worker/             # FRAME = unit inside segment
│   ├── mod.rs
│   ├── types.rs              # FrameInput / FrameOutput
│   ├── encrypt.rs            # EncryptFrameWorker
│   ├── decrypt.rs            # DecryptFrameWorker
│   └── tests.rs              # frame-level tests
│
├── framing/                  # Wire format (NO CRYPTO)
│   ├── mod.rs
│   ├── types.rs              # FrameHeader / FrameRecord / FrameError
│   ├── encode.rs
│   ├── decode.rs
│   └── tests.rs              # framing invariants
│
└── tests/
    └── roundtrip.rs          # Full v2 encrypt → decrypt (single thread)
```

---

## 🧭 Dependency Direction (STREAM_V2)

```text
headers
   ↑
crypto
   ↑
stream_v2
```

## 🧭 Dependency Direction (CRITICAL)

```text
framing
   ↑
frame_worker
   ↑
segment_worker
   ↑
pipeline
   ↑
core
```

✔ **Never inverted**
✔ **No shared mutable state**
✔ **Crypto only appears at frame/segment worker level**

---

## 📦 `mod.rs` (example)

```rust
// src/stream_v2/mod.rs

pub mod core;
pub mod io;
pub mod pipeline;

pub mod framing;
pub mod frame_worker;
pub mod segment_worker;

pub use core::{encrypt_stream_v2, decrypt_stream_v2};
```

---

## 🔐 Segment Worker Responsibilities

| File         | Responsibility                 |
| ------------ | ------------------------------ |
| `types.rs`   | Pure data structures           |
| `crypto.rs`  | Key derivation, nonce strategy |
| `encrypt.rs` | Segment → encrypted wire       |
| `decrypt.rs` | Wire → plaintext frames        |
| `tests.rs`   | Deterministic correctness      |

No IO.
No threads.
No channels.

---

## 🧪 Why This Layout Compiles Fast

* Tests don’t pull in:

  * PyO3
  * Tokio
  * async
* `segment_worker/tests.rs` touches **<15 files**
* `cargo test segment_worker` is seconds after first build

---

## 🧠 Why This Is the *Correct* Architecture

This matches **industrial streaming engines**:

| System      | Unit             |
| ----------- | ---------------- |
| NVMe        | submission queue |
| Kafka       | batch            |
| RocksDB     | SST block        |
| Our system  | **segment**      |

We are **not guessing** — we’re converging on proven design.

---
