# E2E Testing Report — Knafeh v1.0.0

**Date:** 2026-03-29
**Environment:** Linux 6.17, x86_64
**Rust toolchain:** 1.94.1 stable
**Python:** 3.14.2

---

## Transport Architecture

| | Raw QUIC | HTTP/3 |
|--|----------|--------|
| **Library** | quinn 0.11 (pure Rust) | tokio-quiche 0.1 (Cloudflare quiche) |
| **TLS** | rustls (ring) | BoringSSL (C++ FFI) |
| **Protocol** | Direct bidirectional QUIC streams | HTTP/3 on QUIC |
| **Per-request overhead** | ~20 bytes wire framing, 0 mutex locks, 3 await points | ~200+ bytes HTTP/3 headers, 3 mutex locks, 5 await points, 1 task spawn |
| **API** | `QuicClient` / `QuicServer` | `Client` / `Server` |

### Why HTTP/3 is ~3x slower (structural, not fixable)

The HTTP/3 path has inherent per-request overhead from the tokio-quiche event-loop model:
- **3 mutex locks** per call: pool acquire (sync), register_pending (async), pool release (sync)
- **3 async mutex locks** per response in event loop: stream→request mapping, pending check, pending resolve
- **1 task spawn** per response: tokio scheduler overhead for dispatch
- **HTTP/3 header construction**: ~6 pseudo-headers + metadata Strings allocated per request
- **3 linear header scans** on response: status, message, metadata parsed separately

Raw QUIC skips all of this: direct `open_bi()` → `write_all()` → `read_to_end()`.

---

## 1. Transport Comparison — Release Mode (Rust client, Rust server)

### Sequential — 10,000 requests

| Transport | Codec | Throughput | p50 | p90 | p99 |
|-----------|-------|-----------|-----|-----|-----|
| **Raw QUIC (quinn)** | RawProtobuf | **12,680 rpc/s** | **75.5 us** | **99.9 us** | **129.9 us** |
| HTTP/3 (tokio-quiche) | JSON | 4,404 rpc/s | 225.4 us | 266.4 us | 290.5 us |
| HTTP/3 (tokio-quiche) | Protobuf | 4,298 rpc/s | 234.2 us | 269.3 us | 293.0 us |
| HTTP/3 (tokio-quiche) | RawProtobuf | 4,282 rpc/s | 229.7 us | 267.8 us | 313.8 us |

### Parallel — 5,000 requests, 32 in-flight

| Transport | Throughput |
|-----------|-----------|
| **Raw QUIC (quinn)** | **103,546 rpc/s** |
| HTTP/3 (tokio-quiche) | 15,117 rpc/s |

### Key findings

- **Raw QUIC is 3.0x faster sequentially and 6.9x faster in parallel** than HTTP/3.
- **Codec choice has negligible impact** — all three HTTP/3 codecs within 3% of each other.
- Raw QUIC is competitive with bxp-core (12,680 vs 13,249 rpc/s) which uses quinn + Cap'n Proto.

---

## 2. Codec Comparison — Debug Mode (HTTP/3, Rust client → Rust server)

### 100 calls

| Codec | Sequential | Concurrent (50) |
|-------|-----------|-----------------|
| JSON | 535 rpc/s | 1,442 rpc/s |
| Protobuf | 528 rpc/s | 1,460 rpc/s |
| RawProtobuf | 539 rpc/s | 1,457 rpc/s |

Codecs are indistinguishable — the transport layer dominates.

---

## 3. Server Comparison — Debug Mode (Rust client, HTTP/3)

### 100 seq + 50 conc

| Server | Sequential | Concurrent |
|--------|-----------|------------|
| Rust (tokio-quiche) | 535 rpc/s | 1,457 rpc/s |
| Python (tokio-quiche via pyo3) | 398 rpc/s | 866 rpc/s |

Python server is ~25% slower in debug mode due to `spawn_blocking` + GIL overhead for the echo handler. In release mode the gap narrows since the transport layer dominates.

---

## 4. Python Client — Debug Mode (HTTP/3, Protobuf codec)

### 100 calls against Python server

| Client API | Mode | Throughput |
|------------|------|-----------|
| **Python async** | `call_many` (anyio TaskGroup) | **626 rpc/s** |
| Python sync | Sequential | 595 rpc/s |
| Python async | Sequential | 516 rpc/s |

`call_many` with anyio structured concurrency remains the fastest Python path.

---

## 5. Comparison with External Frameworks

| Framework | Transport | Codec | Sequential rpc/s | p50 |
|-----------|-----------|-------|-----------------|-----|
| bxp-core | quinn | Cap'n Proto | 13,249 | 64 us |
| **Knafeh Raw QUIC** | quinn | RawProtobuf | **12,680** | **75.5 us** |
| tonic (gRPC) | HTTP/2 + TCP | Protobuf | 10,543 | 89 us |
| **Knafeh HTTP/3** | tokio-quiche | any | **4,298** | **230 us** |

Knafeh Raw QUIC is within 5% of bxp-core and faster than tonic. The HTTP/3 path trades throughput for gRPC header compatibility.

---

## 6. Test Matrix

| Test Category | File | Count | Status |
|---------------|------|-------|--------|
| Unit tests (codec, router, middleware) | `lib` + `integration_test.rs` | 21 | Pass |
| E2E core (unary, streaming, metadata, middleware) | `e2e_test.rs` | 19 | Pass |
| E2E resilience (retries, errors, backoff) | `e2e_resilience_test.rs` | 6 | Pass |
| E2E load (throughput, burst, mixed, slow service) | `e2e_load_test.rs` | 4 | Pass |
| E2E benchmark (codec + server comparison) | `e2e_bench_test.rs` | 1 | Pass |
| E2E performance (10K+ high-volume) | `e2e_perf_test.rs` | 1 | Pass (ignored in CI) |
| Python binding tests | `test_knafeh.py` | 5 | Pass |
| Python E2E tests | `test_e2e_python.py` | 11 | Pass |
| **Total** | | **68** | **All pass** |
