# ADR 0003: MCP Sampling for Asynchronous Memory Distillation

## Status

Accepted (2026-05-04)

## Context

The hook runtime feature (#7) needs a way to extract durable memory candidates
from session transcripts. Two extraction tiers were considered:

- **Tier 1** — heuristic patterns (self-tag markers, decision phrases, incident
  cues). Always available, but limited recall.
- **Tier 2** — LLM-driven extraction over the drained queue + transcript
  excerpt. Higher recall but needs LLM credentials.

A naive Tier 2 path would have `spool` host its own LLM provider abstraction:
config for API keys, retry policy, model selection, redaction, cost ceiling.
That conflicts with several invariants we already maintain:

- ADR-0001 commits us to a single-user, local-first tool. Holding API keys
  there is friction (every device needs its own setup) and a privacy hazard.
- The hook runtime already runs *inside* an active AI client session. Each
  client (Claude Code, Cursor, Codex, ...) already has a configured LLM
  with its own privacy policy, billing path, and approval UI.
- Expanding `spool` config to include LLM provider settings would push the
  product across the line we drew in ADR-0001 between "memory backbone" and
  "AI provider".

We also do not want hooks to call any LLM directly: hooks must return in
milliseconds, must be silent on failure, and must not surprise the user
with outbound network traffic from a hook script.

## Decision

`memory_distill_pending` (the MCP tool that hooks ultimately drive) performs
Tier 2 extraction by issuing a **`sampling/createMessage` reverse JSON-RPC
request** back to the running AI client over the same stdio channel. The
client's normal sampling/approval pipeline produces the response; `spool`
parses the returned JSON array of `{kind, summary}` candidates and writes
them to the ledger as `candidate` records.

When the client did not advertise `capabilities.sampling` during
`initialize`, or the sampling round-trip fails (rejection, timeout,
transport error), the pipeline records the reason in `fallback_used` and
**always** runs Tier 1 heuristics underneath — sampling is additive, not
load-bearing.

Concretely:

- Hook scripts never invoke an LLM. They write signals to
  `.spool/distill-pending.queue` and call `memory_distill_pending` over
  MCP (or its CLI cousin).
- The MCP server's sampling reverse-call is implemented in
  [`src/mcp.rs`](../../src/mcp.rs) as `McpSamplingClient`, which uses a
  shared writer + pending oneshot map (`StdSamplingChannel`) so the
  reader thread can route inbound responses to the waiting future.
- The pipeline at [`src/distill/pipeline.rs`](../../src/distill/pipeline.rs)
  exposes a `SamplingClient` trait. `McpSamplingClient` implements it for
  production; `NoopSamplingClient` and a fake stub cover unit tests; the
  `spool hook stop` CLI path uses `NoopSamplingClient` because the CLI is
  not running inside an MCP session.
- Tier 1 self-tags become `accepted` (wakeup-eligible immediately).
  Sampling-derived signals become `candidate` (reviewed before promotion).

## Consequences

### Wins

- `spool` holds **zero** LLM credentials. The user's existing AI client is
  the policy boundary for sampling approval, billing, and privacy. This
  preserves ADR-0001's "single-user, local-first" stance.
- Per-client privacy controls are honored automatically: Claude Code's
  approval UX, future MCP clients' allow/deny prompts, and per-session
  rejection all flow through one wire format.
- Tier 1 heuristics give us a deterministic, dependency-free baseline.
  Sampling can be unavailable (capability not declared), refused (user
  rejects), or fail (timeout, parse error) without breaking the pipeline.
- A single async wire format means we do not have to model OpenAI vs
  Anthropic vs Google APIs, retry budgets, model variants, or rate limits
  inside `spool`.

### Costs

- The MCP stdio loop must support reverse JSON-RPC. Implementation cost
  is non-trivial (`StdSamplingChannel`, dedicated writer/reader threads,
  pending-id routing, oneshot timeout) and we now own that surface
  forever.
- Sampling capability is uneven across MCP clients. As of 2026-05, only
  Claude Code is verified end-to-end. Cursor / Codex / others may
  silently fall back to Tier 1 until they implement
  `sampling/createMessage`. Documented in
  [`docs/HOOK_RUNTIME_ROUND_14_PLAN.md`](../HOOK_RUNTIME_ROUND_14_PLAN.md).
- The latency of a sampling round-trip is bounded by
  `SAMPLING_REVERSE_TIMEOUT` (currently 30s). Hooks themselves never
  block on sampling, but the `memory_distill_pending` tool call can.
- Test surface grows: `mcp_smoke` must mock the reverse-call wire
  (`call_tool_with_sampling`) and assert both the success path and the
  rejection-fallback path.

### Non-Goals

- We do not build a spool-side LLM provider abstraction in this round.
  If future work needs offline distillation (no MCP client present), it
  will be a separate ADR.
- We do not promote sampling-derived candidates automatically; promotion
  remains a user action.
- We do not negotiate per-model parameters from the client side beyond
  `maxTokens` and a fixed system prompt; tuning is the client's job.

## References

- ADR 0001 — Local-first developer memory OS
- `src/mcp.rs` — `McpSamplingClient`, `StdSamplingChannel`, `serve_stdio`
- `src/distill/pipeline.rs` — `SamplingClient` trait, `run_with_sampling`
- `tests/mcp_smoke.rs` — `call_tool_with_sampling`, sampling persistence
  and rejection-fallback e2e tests
- `docs/HOOK_RUNTIME_ROUND_14_PLAN.md` — 5-round implementation log
