# ADR-0017: Per-File Diff Splitting for Token Budget Fitting

## Status

✅ Accepted

## Context

omni-dev sends commit diffs to AI models for amendment generation (`twiddle`) and
commit checking (`check`). ADR-0009 established token-budget-aware batch planning,
which groups commits into batches that fit the model's context window and falls back
to solo batches for oversized commits. However, a single commit can contain a diff
that exceeds the entire input context window — a large refactoring touching dozens of
files, or a single generated file with thousands of lines. Solo batching does not help
when the commit itself is the problem.

An earlier implementation addressed this by progressively degrading diff quality
within a single request: truncating the diff at newline boundaries, replacing it with
a `diff --stat` summary, and finally stripping it to a file list. Each level
preserved less context for the AI. Three problems with this approach motivated a
change:

- **Silent quality loss.** The caller had no visibility into which degradation level
  was applied. A `twiddle` invocation might silently produce amendments based on a
  file list rather than the actual changes, yielding superficial or incorrect commit
  messages.

- **Single-request constraint.** The approach assumed a commit's analysis must be
  completed in one API call. Relaxing that constraint opens a better trade-off: more
  requests at full quality versus fewer requests at degraded quality.

- **No file-level structure.** The diff was stored as a single flat blob per commit
  (`{hash}.diff`). Without per-file boundaries, the system could not reason about
  which portions of the diff to include or exclude.

Two alternatives to per-file splitting were considered:

- **Summarisation via a secondary AI call.** Condense the diff with a preliminary AI
  call, then send the summary for analysis. This adds latency and cost, and the
  quality of the summary is itself unbounded — the summarisation call faces the same
  context limit.

- **Hard failure when the budget is exceeded.** Report an error and require the user
  to split their commit manually. This is correct but unhelpful: the tool has enough
  information to split automatically.

## Decision

We will split oversized commit diffs at file and hunk boundaries across multiple AI
requests, preserving full diff content where possible and replacing individual
unsplittable items that exceed the chunk budget with explicit placeholders. The system
will never automatically fall back to stat-only or file-list-only output for an entire
commit.

### Per-file diff parsing

`split_by_file` in `src/git/diff_split.rs` splits a flat unified diff string at
`diff --git a/` boundaries, producing a `Vec<FileDiff>`. Each `FileDiff` holds the
file's path (extracted from the `b/` side of the header), the raw text of that file's
diff section, and its byte length.

`split_file_by_hunk` splits a single `FileDiff` into per-hunk segments at `@@`
boundaries. Each `HunkDiff` includes a copy of the file header (`diff --git`,
`index`, `---`, `+++` lines) so it is self-contained and can be sent to the AI
without additional context.

### Per-file storage

`write_diff_to_file` in `src/git/commit.rs` writes per-file diffs alongside the
existing flat blob. Each commit's diffs directory contains `0000.diff`, `0001.diff`,
etc. — one file per `FileDiff`. `CommitAnalysis` stores these as
`file_diffs: Vec<FileDiffRef>`, where each `FileDiffRef` records the repository-
relative path, the absolute path on disk, and the byte length (from `fs::metadata`,
avoiding content reads during planning).

The flat `{hash}.diff` file is retained for backward compatibility with code paths
that do not require per-file granularity.

### Greedy file-packing algorithm

`pack_file_diffs` in `src/claude/diff_pack.rs` groups a commit's per-file diffs into
chunks that fit within a token budget. The algorithm mirrors `plan_batches` from
ADR-0009:

1. Convert each `FileDiffRef` into a `PackableItem` with an estimated token cost
   derived from `estimate_tokens_from_char_count`.
2. If a file's estimated tokens exceed the chunk capacity, read it from disk and
   split it into per-hunk items via `split_file_by_hunk`. Each hunk item carries a
   `diff_override` containing the pre-sliced content (file header + hunk text). If
   a file has no hunk markers (binary files, mode-only changes) and exceeds the
   capacity, it is replaced with a placeholder item.
3. If an individual hunk still exceeds the chunk capacity after splitting, it is
   replaced with a placeholder item containing the file header and an omission note
   with the byte size and estimated token count.
4. Sort all items largest-first (first-fit-decreasing).
5. Place each item into the first existing chunk with remaining capacity. If no chunk
   fits, create a new one.
6. Apply a 30% headroom factor (`CHUNK_CAPACITY_FACTOR = 0.70`) to absorb YAML
   serialisation expansion. Diff content with special characters, binary markers, or
   escape sequences can expand 20–30% during YAML serialisation, which is
   significantly more than the 10% headroom used by batch planning
   (`BATCH_CAPACITY_FACTOR = 0.90`). The divergence is intentional: batch planning
   estimates at the commit level from `stat`-based byte counts, while chunk packing
   operates on actual diff content that undergoes YAML block-scalar serialisation.

Placeholder items use a compact format that identifies the omitted file or hunk,
records its byte size and estimated token count, and states that it was too large to
include. This preserves the file's presence in the analysis (the AI knows the file
was changed and how large it is) without exceeding the chunk budget. The placeholder
approach replaces the earlier hard-error strategy, which caused the entire commit to
fail when a single unsplittable item exceeded the budget.

### Amendment split dispatch

`generate_amendment_for_commit` in `src/claude/client.rs` first attempts the full
diff in a single request via `try_full_diff_budget`. If the budget is exceeded and
`file_diffs` is populated, it calls `generate_amendment_split`, which:

1. Computes an effective chunk capacity by subtracting known prompt overhead from
   `available_input_tokens`. The overhead includes the system prompt token count,
   the `RepositoryViewForAI` envelope (150 tokens), per-commit metadata (120
   tokens), user prompt template text (1000 tokens), and the commit's own
   `original_message` and `diff_summary`. These constants are shared with the batch
   planner (ADR-0009) to ensure consistent capacity accounting. The resulting
   capacity reflects the budget available exclusively for diff content.
2. Calls `pack_file_diffs` with the effective capacity to produce a
   `CommitDiffPlan`.
3. For each chunk, builds a partial `CommitInfoForAI` containing only that chunk's
   file diffs (using `from_commit_info_partial_with_overrides` to load subset diffs
   from disk or apply hunk overrides).
4. Sends one AI request per chunk, collecting a partial `Amendment` from each.
5. Runs `merge_amendment_chunks`, an AI reduce pass that receives all partial
   proposed messages plus the commit's `diff_summary` and original message, and
   synthesises a single final `Amendment`.

If any chunk request fails, the entire commit fails — the error propagates for
interactive retry at the caller level, matching the existing split-and-retry pattern
from ADR-0009.

### Check split dispatch

`check_commit_split` follows the same structure for commit checking, including the
overhead subtraction step for effective capacity. After collecting
`CommitCheckResult` from each chunk, it merges deterministically:

- **Issues:** union of all `CommitIssue` lists, deduplicated by
  `(rule, severity, section)`.
- **Pass/fail:** `passes` is `true` only if all chunks pass.
- **Suggestion/summary:** an AI reduce pass runs only when at least one chunk
  returned a suggestion; otherwise the first non-`None` summary is used.

### Integration with batch planning

The batch planner from ADR-0009 remains the first line of defence for multi-commit
overflow. `estimate_commit_tokens` in `src/claude/batch.rs` now sums per-file
`byte_len` values from `file_diffs` when available, falling back to `fs::metadata` on
the flat diff file for data produced before per-file storage was introduced. Oversized
solo batches that previously relied on progressive diff reduction now trigger split
dispatch at request time instead.

The split-and-retry logic in `twiddle` and `check` (splitting a failed multi-commit
batch into individual commits) is unchanged. Split dispatch operates one level below:
it handles the case where a single commit's diff exceeds the budget after batch
splitting has already isolated it.

## Consequences

**Positive:**

- **Full diff quality is preserved for most files.** The AI sees the complete
  line-level diff content for every file that fits within the chunk budget. Files or
  hunks that individually exceed the budget after splitting are replaced with
  explicit placeholders that record the file path, byte size, and estimated token
  count. The AI still knows the file was changed and can use the `diff_summary` for
  context; only the line-level detail is omitted.

- **No silent degradation.** Placeholder substitution is logged at `debug` level and
  is visible in the diff content sent to the AI (the placeholder text states what
  was omitted and why). The system never falls back to stat-only or file-list-only
  output for an entire commit. A single oversized file results in a placeholder for
  that file while all other files retain full diff content.

- **Graceful scaling.** A commit touching one hundred files produces dozens of chunk
  requests automatically. The number of requests scales with the commit's size, not
  with a fixed reduction strategy.

- **Per-file storage enables efficient planning.** `FileDiffRef.byte_len` provides
  per-file cost information from `fs::metadata` without reading file contents. Both
  the batch planner and the packing algorithm use this for lightweight estimation.

**Negative:**

- **More API calls for large commits.** A commit that would previously have been
  analysed in one request with a truncated diff now produces N requests at full
  quality plus a merge pass. This increases API cost and latency proportionally to
  the commit's size.

- **Merge pass is approximate.** The amendment merge pass sees partial proposed
  messages and the `diff_summary` but not the full diffs. The synthesised message may
  miss cross-file relationships that would be visible in a single full-context
  request.

- **Disk overhead from per-file diff files.** Each commit's diffs directory now
  contains one file per changed file, in addition to the flat blob. For a commit
  touching fifty files, this adds fifty small files to the scratch directory. The
  scratch directory is ephemeral and cleaned up after the session.

- **Hunk splitting is a last resort with limited context.** When a single file's diff
  is split into hunks, each chunk sees only a subset of the file's changes. The AI
  cannot reason about interactions between distant hunks within the same file.

**Neutral:**

- **The flat diff file is retained.** Code paths that operate on the whole-commit
  diff (e.g. YAML serialisation of `diff_content`) continue to use the flat blob.
  Per-file storage is additive rather than replacing the existing format.

- **`DiffDetail` and the progressive reduction methods have been removed.** The
  `Truncated`, `StatOnly`, and `FileListOnly` variants and their corresponding
  `truncate_diffs`, `replace_diffs_with_stat`, and `remove_diffs` methods no longer
  exist. The only remaining budget-fitting path is: full diff → split dispatch →
  placeholder for unsplittable oversized items.
