# Inkhaven 1.2.20 — Bugfix, consolidation & hardening

_Theme: pay down the debt the 1.2.19 revision cycle
exposed.  No new headline feature — fix the bugs the
duplication audit surfaced, collapse the copy-paste
into shared helpers, then burn down the highest-value
polish carryovers.  The cheapest time to consolidate
the stemmer / book-walk / AI-stream foundations is
**before** the next feature cycle stacks more copies on
them._

Three phase groups: **B** (bugs), **D** (dedup /
consolidation), **H** (hardening / polish).

---

## Phase B — Bugs surfaced by the audit

### B.1 — Russian `ё`-fold missing in three stem surfaces

**The bug.**  1.2.19's revision detectors (`echo`,
`tension`, `continuity_bible`) fold `ё` → `е` before
Snowball stemming, because the Russian algorithm treats
the two as unified and real text uses them
interchangeably.  Three *older* surfaces stem Russian
**without** that fold and so mis-group any `ё`-word:

* `tui/concordance.rs` (`Ctrl+B Shift+L` concordance
  modal) — lines 187, 216.  `пошёл` won't group with
  `пошла`; `зелёный` splits from `зелёные`.
* `tui/style_warnings.rs` (filter-word + repeated-phrase
  overlays) — lines 104, 165, 252.  Filter words and
  repeated phrases containing `ё` silently miss.
* `tui/lexicon.rs` (character / place **name**
  highlighter) — line ~145.  A name like `Алёша`,
  `Фёдор`, `Пётр`, `Семёнов` fails to light up when the
  buffer and the name-list disagree on `ё`/`е` spelling.
  (The module's own doc comment advertises Russian
  inflection matching — this is a correctness gap in a
  shipped feature.)

**The fix.**  Route all six stem sites through the
single shared normalize helper introduced in **D.1**,
which always folds.  B.1 ships the *behaviour* fix; D.1
is the *structural* change that guarantees it can't
regress.  Lands together.

**Tests.**  A Russian regression test per surface: a
`ё`-word and its `е`-spelled inflection must land on the
same stem key / fire the same overlay / highlight the
same name.

### B.2 — Chapter-walk drift audit

The ~8 chapter-prose collectors (see D.3) have already
diverged: `epub` skips Image nodes, `audiobook` strips
footnotes, `manuscript` does neither, and the
subchapter-recursion branch exists in some walks but not
others.  Before D.3 unifies them, enumerate the
intended behaviour (recurse subchapters? strip
footnotes? include image alt-text?) so the consolidated
helper is a deliberate superset, not whichever copy
happened to win.  Output: a short decision table in the
implementation log; the unified helper in D.3 implements
it.

---

## Phase D — Consolidation

Eight duplication categories from the audit, ordered by
value.  Each lands as a shared helper plus deletion of
the copies; net negative LOC, fewer drift surfaces.
Every consolidation keeps the existing public behaviour
(except B.1, which is the one intended behaviour change).

### D.1 — `text::normalize_stem` (fixes B.1)

One helper: `lowercase → ё-fold → optional Snowball
stem`, returning the stem key.  Replaces **5** inline
copies (echo, tension ×2, continuity_bible — correct;
concordance ×2, style_warnings ×3, lexicon — buggy,
fixed by routing through it).  New `src/text.rs` (or
fold into `config.rs` beside `parse_stemmer_language` /
`built_in_stop_words`).  The canonical body already
exists in `echo.rs:116`.

### D.2 — `cli::book_select` (resolve_user_book ×8)

Identical user-book resolution copied across `build`,
`event`, `manuscript`, `export_timeline`, `epub`,
`audiobook`, `continuity`, `tension`.  One helper +
`user_book_chapters`.

### D.3 — `cli::chapter_text` (chapter-prose collection ×8)

The near-identical subtree walks (`doctor_scan` ×2,
`continuity`, `tension`, `epub`, `audiobook`,
`manuscript`, `ai_impl`).  One helper parameterised by
output flavour (plain prose / raw typst) and the
behaviour table from **B.2**.  Highest drift risk of the
set — this is where the inconsistencies already live.

### D.4 — `ai::stream::collect_blocking` (run_blocking ×5)

The `spawn_chat_stream` → `blocking_recv()` token-collect
loop copied in `prompts`, `ai`, `show_dont_tell`,
`continuity`, `tension`.  One `ai` helper returning
`Result<String>`.

### D.5 — shared typst-prose strip (×3)

`strip_leading_heading` + `split_blocks` in `audiobook`,
`epub`, `assemble`.  Unify into a typst-prose module the
reader-exports share.  (Keep `typst_to_plain` vs
`typst_to_xhtml` distinct — those are genuinely
different renderers; only the *block-splitting* and
*heading-strip* are shared.)

### D.6 — one `is_scene_break` home

`manuscript.rs:78` (pub) and `tui/app.rs:1411` (private,
identical).  Keep the `manuscript` one, point `app` at
it.

### D.7 — reading-time + sentence-split unify (optional)

Lower value, do if time allows:

* Reading-time: `text_utils::format_reading_time`
  (fixed 250 wpm) vs `tui/reading_time` (configurable).
  Unify on the configurable one; the fixed caller passes
  250.
* Sentence splitting: `continuity.rs:287` vs
  `sentence_rhythm.rs:272` — share the boundary logic,
  keep the distinct return shapes as thin wrappers.

---

## Phase H — Hardening & polish

The highest-value, lowest-risk carryovers from the
scratchpad.  Pick what fits the cycle after B+D land.

### H.1 — R.3.b `paragraph-too-long` doctor class

A paragraph whose read time exceeds a threshold at
`reading_wpm`.  Info, no autofix — author-judgment
finding.  Naturally consumes the unified reading-time
helper from **D.7**.

### H.2 — Phase G remainder (1.2.15 carryover)

* Uncommitted-changes warning at TUI exit.
* Disk-space pre-flight (needs `fs2` / `sysinfo` — one
  new dep; weigh against the zero-new-dep streak).

### H.3 — C.1.b echo overlay

Live editor overlay for the echo detector, pairing with
the `Ctrl+B Shift+F` style-warning toggle — underline
echoed words in the open paragraph as you write.  Builds
directly on the now-consolidated `text::normalize_stem`.

### H.4 — M1 / M3 security hygiene (1.2.15 carryover)

* Crash-report action-ring detail audit.
* `[script: <slug>]` prefix on script-opened modals.

### H.5 — candidates if scope allows

* C.2.b `bootstrap-continuity <lang>` CLI (the LLM-seed
  command; defer the ru/de lexicon *bundling* until
  native-speaker verification).
* R.1.b ePub cover image + inline images.
* GitHub Release backfill for 1.2.11–1.2.19.

---

## Out of scope (deferred to a feature cycle)

Dialogue & voice, genre packs, AI maturity, finishing &
submission — the feature themes from the 1.2.20
brainstorm.  This cycle deliberately stays
infrastructure-only so the next feature cycle builds on
deduplicated foundations.

## Implementation log

### B.1 — Russian `ё`-fold (landed)

Folded `ё`→`е` before Snowball stemming at all nine
production stem sites across the three affected
surfaces:

* `tui/concordance.rs` — the stop-word `normalise`
  closure + the token grouping key.  The displayed
  surface form keeps its `ё` (fold applies to the
  grouping key only).
* `tui/style_warnings.rs` — all six sites
  (FilterWords init + detect, repeated-phrase stop +
  token, show-don't-tell init + detect).
* `tui/lexicon.rs` — the single `stems_for` helper,
  which folds both the compiled name list and the
  buffer tokens.

Three Russian regression tests (one per surface) assert
a `ё`-word and its `е`-spelled form now share a stem
key / fire the same overlay / highlight the same name.
Suite: 1121 → 1124, all green.  Shipped standalone
ahead of the D.1 `text::normalize_stem` consolidation.

### D.1 — `text::normalize_stem` (landed)

New `src/text.rs` with two helpers:

* `fold_lower(word)` — lowercase + `ё`→`е` fold.
* `normalize_stem(word, &Option<Stemmer>)` — `fold_lower`
  then Snowball-stem when a stemmer is active.  The
  canonical token→key reduction.

Routed every single-stemmer site through it: `echo`
(`stem_of`), `tension` (`significant_stems` +
stop-set), `continuity_bible` (`normalise`),
`concordance` (stop closure + token key, surface form
still untouched), `style_warnings` (all six).  `lexicon`
keeps its multi-algorithm `stems_for` but now folds via
`text::fold_lower`.  The fold now lives in exactly one
place — B.1 cannot silently regress again.  3 new
helper unit tests; suite 1124 → 1127, all green, no new
warnings.  Net deletion of the inline match-arm copies.

### D.6 — one `is_scene_break` home (landed)

Deleted the editor's private copy in `tui/app.rs` (a
37-line duplicate, doc + body) and pointed it at the
existing `crate::manuscript::is_scene_break` via a `use`.
The two impls were behaviour-identical (trim → empty/`§`
guards → strip internal whitespace → 3+ copies of one of
`* - _ ~ #`).  `app.rs` keeps its richer edge-case tests
(`= Chapter`, `***bold***`, `*****`, `  * * *  `), which
now guard the shared function.  `manuscript`'s doc gains
the typst-heading-rejection note.  Suite still 1127.

### D.2 — `cli::resolve_user_book` (landed)

Six CLI commands (build, audiobook, epub, manuscript,
event, export-timeline) each carried a near-identical
user-book resolver — but in two flavours: owned
`Result<Node>` via `Error::Store` (the first four) and
borrowed `Result<&Node>` via `anyhow!` (the last two).

Replaced all six with one `pub(crate) fn
cli::resolve_user_book(h, book_name, context) ->
Result<&Node, String>`: a pure resolver that borrows and
returns a ready-to-display, command-prefixed error
string.  Each caller maps the result into its own world —
`.map_err(Error::Store)?.clone()` for the owned/Store
commands, `.map_err(|m| anyhow!(m))?` for the borrowed/
anyhow ones.  The error text is now uniform (and lists
available books on a miss, the most informative of the
old variants).  Net −215 LOC across the seven files;
suite still 1127, no new warnings.

### B.2 — chapter-walk divergence audit (decided)

Eleven collectors, mapped before unifying.  They split
into **two source-worlds** and three output flavours:

| Collector | Source | Output |
|---|---|---|
| continuity / tension `chapter_prose` | fs | raw |
| doctor `read_chapter_prose` | fs | raw |
| doctor `collect_chapter_paragraph_prose` | fs | plain |
| manuscript `gather_paragraphs` | fs | plain (trimmed) |
| epub `append_branch_prose` | store | xhtml + `<h2>` |
| audiobook `append_branch_prose` | store | plain |
| ai_impl `collect_chapter_bodies` | store | raw |

Decisions:

1. **Two source-worlds stay by design.**  Filesystem
   collectors read the on-disk `.typ` projection (the
   narrative scanners + manuscript); store collectors
   read bdslib content directly (reader-exports + live
   editor).  Folding them forces a source change for one
   side — out of scope for a pure dedup.  D.3 unifies
   *within* the filesystem world.
2. **Canonical traversal = `collect_subtree`** + filter
   `Paragraph`.  manuscript's manual recursion becomes
   this (a strict superset: it also catches paragraphs
   nested under non-Subchapter containers).
3. **Canonical primitive = raw paragraphs.**  Output
   transforms (plain / raw-join) and per-paragraph trim /
   empty-drop stay at call sites, where they differ by
   consumer.
4. epub's `<h2>` subchapter-heading injection stays
   epub-only (an xhtml structural concern the flat-subtree
   primitive can't express).

### D.3 — `cli::book_walk` (landed)

New `src/cli/book_walk.rs`:

* `chapter_paragraphs_raw(layout, h, chapter_id)` — each
  paragraph's raw typst, subtree flattened, missing /
  unreadable files skipped.
* `chapter_raw_prose(...)` — the same joined per-paragraph
  with newlines, the blob the scanners consume.

Migrated the five filesystem collectors: continuity /
tension `chapter_prose` and doctor `read_chapter_prose`
→ `chapter_raw_prose`; doctor
`collect_chapter_paragraph_prose` and manuscript
`collect_chapters` rebuilt on `chapter_paragraphs_raw` +
their own `typst_to_plain` / trim / drop-empty.  The
three store collectors (epub, audiobook, ai_impl) stay
per B.2; epub + audiobook still share an
`append_branch_prose` shape (store-side) — noted as a
**D.3.b** follow-up (they differ: xhtml+`<h2>` vs plain).
Net −89 LOC; suite 1127, no new warnings.  The extracted
walk is behaviour-identical to the originals and stays
covered by the existing caller tests.

### D.4 — `ai::stream::collect_blocking` (landed)

Four CLI commands (continuity, tension, show-dont-tell,
prompts) each inlined the same blocking drain of
`spawn_chat_stream`: collect every token into a String,
emit a `.` to stderr per token, break on `Done`, bail on
`Error`.  Extracted into
`ai::stream::collect_blocking(client, model, system,
prompt) -> Result<String, String>` (one-shot; empty
history; `Err` is the raw inference-error message, no
prefix).

continuity / tension delegate from their `run_blocking`
wrappers (mapping the error to `Error::Store`);
show-dont-tell / prompts match on the result and keep
their own report-and-`return Ok(())` policy.  The helper
owns the spawn + drain + progress dots; callers keep all
I/O policy (trailing newline, error formatting).

The fifth `run_blocking`-family site, `cli::ai`, stays as
is: it streams tokens **live to stdout** without
collecting — an interactive passthrough, a genuinely
different mode, not a fit for `collect_blocking`.  Net
−72 LOC; suite 1127, no new warnings.

### D.5 — `typst_prose` strip/split (landed)

ePub and audiobook carried byte-identical
`strip_leading_heading` (drop a leading `= …` org title +
following blanks; keep `==` subheadings) and
`split_blocks` (blank-line-separated paragraphs).
Extracted both into a new `src/typst_prose.rs`
(`pub(crate)`), and moved ePub's three strip tests there +
added two `split_blocks` tests as the canonical coverage.

`crate::assemble`'s `strip_leading_heading` stays
separate, per the audit: it matches **any** `=`-prefixed
line (including `==` section headings) and is
whitespace-aware, because it feeds typst compilation, not
reader-export prose — folding it in would change its
behaviour and break its own tests.  Net −91 LOC; suite
1127 → 1129 (−3 moved ePub tests, +5 in the new module),
no new warnings.

## Phase H — Hardening & polish

### H.1 / R.3.b — `paragraph-too-long` doctor class (landed)

New `ScanClass::ParagraphTooLong`: flags a paragraph
whose estimated read time at `editor.reading_wpm` exceeds
the new `editor.paragraph_long_secs` setting (default
180s ≈ 600 words at 200 wpm).  Info severity, no autofix
— a wall of text can be a deliberate run-on.  Runs by
default (not opt-in).

Wired through the full `ScanClass` pattern (variant, slug
+ from_slug round-trip, `ALL`, `scan_project` dispatch,
the author-judgment `apply_fix` arm) plus
`scan_paragraphs_too_long`, which reuses the D.3
`collect_chapter_paragraph_prose` walk and
`progress::count_words`, computing read time with the
same `words × 60 / wpm` the editor's reading-time chip
uses (so a finding matches what the writer sees).  New
config field + default + `CONFIGURATION.md` entry.  Suite
1129, the slug round-trip test auto-covers the new class;
no new warnings.

### H.2a / Phase G — low-disk pre-flight (landed)

New `crate::disk` (best-effort `fs2` wrapper):
`available_bytes`, `is_low(path, threshold_mb)`, `human`.
The TUI runs a one-time pre-flight at startup — when the
project's volume has less than `editor.disk_warn_mb`
(default 100 MiB) free, the status line warns before a
session / long export.  Atomic writes already fail safely
on a full disk; this is the proactive half.

Per the user's call, this uses a real dependency
(`fs2` — tiny, cross-platform statvfs/GetDiskFreeSpaceEx
wrapper) rather than a fragile `df` shell-out — the one
deliberate new dep this cycle.  4 unit tests; config field
+ `CONFIGURATION.md` entry.  Suite 1129 → 1133.

### H.2b / Phase G — uncommitted-changes quit guard (landed)

New `crate::git::uncommitted_count(root)` shells out to
the user's own `git status --porcelain` (zero
dependency; `None` when not a repo / git absent).  On
quit, after the open buffer autosaves, `request_quit`
checks it: if `editor.warn_uncommitted_on_exit` (default
true) and the working tree has changes, a
`ConfirmQuitUncommitted { count }` modal asks before
exiting — Y quits anyway, N stays.  Closes the Phase G
remainder's exit-warning item.  Config field +
`CONFIGURATION.md` entry; 1 unit test (non-repo → None).
Suite 1133 → 1134.

### H.3 / C.1.b — live echo overlay (landed)

The inline editor companion to the C.1
`echo-repetition` doctor scan.  `Ctrl+B Shift+K` (new
`editor.echo_overlay` default + session toggle, mirroring
the Shift+F style overlay) underlines, in the open
paragraph, every occurrence of a word echoing across
nearby paragraphs — caught as you write.

New `tui::echo_overlay::EchoHighlighter` is the cheap
per-line half (a stem set + `text::normalize_stem`,
mirroring `FilterWordsDetector`).  The cross-paragraph
half — the hard part flagged in the audit — lives in
`App::refresh_echo_overlay`: it finds the open
paragraph's chapter, gathers the chapter's paragraphs
from the store (open one live, including unsaved edits),
runs `echo::detect_echoes`, and caches the stems whose
window covers the open paragraph.  Two-level caching
keeps it cheap: siblings re-gathered only on paragraph
navigation (store I/O), the echo pass skipped entirely
when the open text is unchanged (content-hash key) —
no per-keystroke I/O.  Rendered through the existing
`StyleHit` / `build_row_spans` path via a new
`StyleWarningKind::Echo` (sharing the repeated-phrase
colour — a dedicated theme colour is a follow-up).

Keybind + label + description + `KEYBINDING.md` +
`CONFIGURATION.md`.  3 unit tests; suite 1134 → 1137.

## Phase R — Reader-experience polish

### R.1.b (echo colour) — dedicated echo-overlay colour (landed)

C.1.b shipped the echo overlay sharing the
repeated-phrase magenta — so a cross-paragraph echo and
a within-paragraph repeated phrase rendered identically.
Gave the overlay its own colour: `theme`-config
`style_warning_echo_fg` + `_modifier` (serde-default;
older project configs keep working, empty falls back via
`color_or` to a muted purple `#b48ead` distinct from the
repeated-phrase magenta).  `Theme` resolves both,
`set_by_name` recolours at runtime (the `ink.theme.set`
script path), and the `highlight.rs` Echo arm paints the
new field instead of borrowing repeated-phrase.  Docs +
4 theme tests (distinct default, empty-fallback, runtime
recolour).

### R.1.b (cover) — EPUB cover image (landed)

`inkhaven epub` now embeds a cover by convention: a
`cover.png` / `cover.jpg` / `cover.jpeg` next to
`inkhaven.hjson` is auto-detected (`cli::epub::detect_cover`,
png-first, empty/unreadable skipped with a warning, never
fatal).  `EpubMeta` gains an optional `cover: Option<EpubCover>`;
when present the writer stores the image verbatim
(`OEBPS/cover.<ext>`, not re-deflated) + a full-page
`cover.xhtml`, and `content_opf` emits the EPUB3
`properties="cover-image"` manifest item, the EPUB2
`<meta name="cover">` back-compat tag, and a leading
`linear` spine entry so the cover is both the library
thumbnail and the opening page.  `None` keeps the R.1
text-only output byte-identical (asserted by
`no_cover_keeps_opf_cover_free`).  8 new tests (epub +
cli); reader-exports tutorial documents the convention.

## Phase U — User-global config layering

### U.1 — config cascade (`~/.config/inkhaven`) (landed)

Research outcome: HJSON has no `include`/`import` directive and
`serde-hjson` adds none; a textual include preprocessor would be
fragile.  Implemented the user's fallback instead — a config
**cascade** via deep value-merge, which is strictly better than an
include.

`Config::load_layered(project_path)` layers, lowest → highest:
built-in defaults → project `inkhaven.hjson` →
`~/.config/inkhaven/config.hjson` → `~/.config/inkhaven/conf/*.hjson`
(sorted).  The global files are **partial** (only the keys present
override) and **win over the project** — deliberately, because
`inkhaven init` writes a *full* config, so a project-wins cascade
would mask the user's global preferences.

Mechanism: each HJSON file parses to a `serde_json::Value`
(`serde_hjson::from_str::<Value>`), the values deep-merge
(`merge_value`: objects merge key-by-key, everything else replaces),
and the merged value deserialises once into `Config` — starting from
`to_value(Config::default())` so the result is always complete.
Global dir honours `$XDG_CONFIG_HOME`, else `~/.config`.  A malformed
*global* file is skipped with a WARN (never bricks every project); a
malformed *project* file stays fatal, like `load`.

Wired through all 47 project-context `Config::load(&layout.config_path())`
sites (TUI startup + every CLI command) → `load_layered`.  The pure
`load` stays for the in-app config-editor reload, `init`, and the
prompts/tts paths (raw-file semantics; also keeps the test suite
hermetic — `load_layered_from` takes an injected dir so tests never
touch the real `~/.config`).  11 unit tests (merge, file discovery +
ordering, global-wins, fall-through, conf precedence, malformed-skip,
partial-project).  CONFIGURATION.md gains a "Global overrides"
section.  Suite 1148 → 1159.

### U.1.b — 15 color-style presets (landed)

`color_styles/` ships 15 popular schemes as partial `theme` overrides
that showcase the U.1 cascade — 9 dark (Dracula, Nord, Gruvbox Dark,
Solarized Dark, Tokyo Night, Monokai, One Dark, Material Ocean,
Catppuccin Mocha) + 6 light (Solarized Light, Gruvbox Light, GitHub
Light, Catppuccin Latte, One Light, Ayu Light).  Each maps the
scheme's palette onto the full Inkhaven semantic field set; drop one
into `~/.config/inkhaven/conf/` to theme every project.  `README.md`
indexes them with usage.  A `color_style_presets_all_parse` test
validates every file is valid HJSON, contains only parseable theme
colours, and layers into a complete Config — guarding the presets
against field-rot.  Suite 1159 → 1160.
