# AR-scanner: how grund discovers declarations and citations

The scanner is the single tree-walk that produces all of grund's input data. Every check in [§FS-check](../functional-spec/FS-check.md#fs-check-grund-validates-every-reference-in-a-repo) and every retrieval in [§FS-show](../functional-spec/FS-show.md#fs-show-grund-reads-a-single-declaration-body-by-id) derives from what the scanner finds. Speed ([§GOAL-fast-feedback](../goals.md#goal-fast-feedback-grund-must-be-as-fast-as-possible)) is set here.

## 1. Tree walk

Recursive walk from a root path using the `ignore` crate (the same walker that powers `ripgrep`). The walker is chosen specifically because it gives us `.gitignore` support for free — see 1.1 below.

Directory-level skip rules:

- Hidden directories (any name starting with `.`) are skipped — this already covers `.next`, `.venv`, and friends.
- Build/output directories named in the skip list (`target`, `node_modules`, `.git`, `dist`, `build`, `.venv` by default — [§FS-config.3.5](../functional-spec/FS-config.md#35-scan--what-gets-walked)) are skipped at any depth.
- The skip list is configurable per [§GOAL-configurable](../goals.md#goal-configurable-every-default-is-overridable) and [§FS-config.3.5](../functional-spec/FS-config.md#35-scan--what-gets-walked).

Files are filtered by extension to those that can plausibly contain specs or inline declarations: `.md` and a curated list of source-file extensions.

### 1.1 Respecting `.gitignore` and friends

By default, the walker honors every form of ignore file the `ignore` crate recognizes:

- `.gitignore` files at any depth (nearest-wins precedence, as `git` itself does).
- `.git/info/exclude`.
- The global `core.excludesFile` configured in `git config`.
- `.ignore` files (the ripgrep convention) for `grund`-specific exclusions that are not appropriate for `git`.

This means `grund` will not scan files that `git` would not commit. Generated artefacts, secrets, and vendored dependencies are skipped automatically without any `grund.toml` configuration. A repo's existing `.gitignore` is the source of truth.

The behavior is overridable via `[scan] respect_gitignore` in `grund.toml` (default `true`). Set to `false` only when you genuinely need to scan ignored paths — e.g., a repo that commits both `node_modules/` and its own specs in unusual layouts.

The directory-level skip lists in 1 above are applied **in addition** to ignore-file rules, never instead of them.

## 2. Per-file scan

A single linear pass over each file's lines, performing three jobs simultaneously:

### 2.1 Declaration detection

A regex matches declaration lines in one of two context-specific shapes:

1. **Markdown-form** — `#{1,N} <ID>[:…]`: a `#`-prefixed heading at any markdown level. This is how `.md` files declare (`# FS-foo:`).
2. **Code-form** — `<comment-prefix> <ID>[:…]`, or bare `<ID>[:…]` inside a Python docstring: a doc-comment line with the ID directly after the marker, no markdown `#` prefix. Decided in [§DF-code-declarations-drop-hash](../decisions/functional/DF-code-declarations-drop-hash.md#df-code-declarations-drop-hash-code-resident-declarations-may-drop-the--prefix). The comment prefix is **required** outside Python docstrings — without a `#` heading in markdown or a doc-comment marker in source, a line `FS-foo: anything` in prose is not a declaration.

`<ID>` is the configured `[id]` grammar ([§FS-config.3.2](../functional-spec/FS-config.md#32-id--id-grammar)) with `{kind}` drawn from a configured `[[kinds]]` prefix. The heading may sit at any markdown level when written in markdown-form: file-form `GND`/`FS`/`AR`/`DF`/`DA` declarations are H1 (`# FS-… :`), and `GOAL` and `RM` declarations are H2 inside `docs/goals.md` and `docs/roadmap.md` respectively. Code-form declarations are treated as level 1 *within* the comment block.

When the regex matches, the line opens a new "current declaration" context and the **declaration heading level** `L` is recorded:

- Markdown-form: `L` is the count of `#` on the line (`#` -> 1, `##` -> 2, ...).
- Code-form: `L` defaults to `1`. The declaration is conceptually a "level-1" heading inside the doc-comment block — its sections are still numbered `## 1. …`, `### 1.1 …`, etc., one or more `#` deeper than the declaration line.

Both forms record the same `Declaration` struct downstream; consumers (`grund <ID>`, `grund check`, `grund refs`) do not care which shape the source used. (`E2E` declarations are the exception — they are directories, not heading lines; see §6.)

### 2.2 Section detection

Within a declaration context whose heading is at level `L`, a numbered subsection heading is a line of the form `#{L+1,} <n₁.n₂.….n_d>[.] <title>` — at least one `#` more than the declaration heading, then a dotted number of one or more components, an **optional** trailing `.`, whitespace, and the heading text. The line is recorded on the current declaration as the section path `n₁.n₂.….n_d` together with its `<title>` text, source line, and Markdown heading level (the heading text is needed by [§FS-fmt.6](../functional-spec/FS-fmt.md#6-cross-reference-emission) / [§DF-md-link-anchor-strategy](../decisions/functional/DF-md-link-anchor-strategy.md#df-md-link-anchor-strategy-heading-text-slugs-re-derived-on-every-fmt-pass) and by `grund <ID> --format=md`; the source line and level are needed by [§FS-check.3.9](../functional-spec/FS-check.md#39-section-heading-level-mismatch)). The dotted number fixes the section's tree position, and the configured `[id] section_heading_levels` mode fixes how strictly the written `#` depth must match it ([§FS-config.3.3](../functional-spec/FS-config.md#33-section-paths--arbitrary-nesting-depth)). In `"strict"` mode (the default), the heading level must be exactly `L + d`, where `d` is the number of dotted path components: under an H1 declaration, `## 1. Inputs`, `### 1.1 Recognized citations`, and `#### 3.1.2 Details` are consistent, while `## 1.1 Recognized citations` is recorded but later reported as a check error. `"warn"` records the same mismatch as a warning. In `"loose"` mode, the historical rule applies: the `#` count only has to be deeper than the declaration heading, so `## 1.1` and `### 1.1` both declare section `1.1`. Plain, unnumbered headings and bold labels are just Markdown prose structure and are not recorded as sections. Nesting depth is unbounded ([§FS-config.3.3](../functional-spec/FS-config.md#33-section-paths--arbitrary-nesting-depth)); the recorded set is what [§AR-checker.2.3](../../crates/grund-core/src/checker.rs) validates citations against.

### 2.3 Citation detection

The citation regex matches the configured marker ([§DF-reference-marker](../decisions/functional/DF-reference-marker.md#df-reference-marker-use--as-the-reference-marker-with--as-the-typing-trigger); default `§`) immediately followed by an `<ID>` token, with an optional `<sep><section-path>` suffix, anywhere in the file. In default (non-strict) mode it **additionally** matches bare ID tokens — but, in source files (every extension except `md`), a bare token whose start column lies inside a string literal is **not** treated as a citation, applying the same deterministic left-to-right quote-tracking rule as [§FS-fmt.2.3.1](../functional-spec/FS-fmt.md#231-string-literal-exclusion-rule). This keeps an ID-shaped substring inside a runtime string from raising a false dangling-ref. Marker-prefixed citations are recognized regardless of string context — a `§` in a string is a deliberate citation. Markdown files have no string literals, so the carve-out does not apply there. In `[reference] strict = true` mode only marker-prefixed citations are recognized at all. A declaration's own heading line is never counted as a citation of the ID it declares.

## 3. Output

The scanner produces a `Findings` struct containing:

- `declarations: BTreeMap<Id, Vec<Declaration>>` — keyed by ID, with file/line, stub-info, and the recorded sections (each section path paired with its heading text — §2.2) per declaration. An `E2E` declaration (§6) carries its case-directory path, fixture list, invocation, and expected exit code instead.
- `citations: Vec<Citation>` — each with the referenced ID, optional section, file, line, and start column, plus whether it was written marker-prefixed or bare.

This is the only structured output the scanner produces. Everything downstream (checking, showing, IDE diagnostics) operates on this data structure.

## 4. Inline declarations in language doc-comments

The scanner is designed so that an inline declaration — most commonly an `AR-NNN-<slug>` for an architectural spec — can live inside the **class, method, module, or package doc-comment** of any major language. This makes class-level documentation a first-class place to put architecture specs: the spec body sits with the code it describes, and a stub under `docs/architecture/` points at it through a single-line H1 of the form `# <ID>: [<path>](<path>)`.

The recognized doc-comment forms (matched as comment prefixes preceding the heading line):

| Language(s)              | Doc-comment form                                  | How the regex sees it                |
|--------------------------|---------------------------------------------------|--------------------------------------|
| Java, Kotlin, Scala      | `/** … */` (Javadoc / KDoc / Scaladoc)            | `/*` opens; ` * ` on continuation    |
| C, C++                   | `/** … */` (Doxygen) or `/// …`                   | `/*` or `//` (covers `///`)          |
| C#                       | `/// <summary>…</summary>` (XML doc)              | `//` (covers `///`)                  |
| Rust                     | `/// …` outer, `//! …` inner, `/** … */` block    | `//` covers `///` and `//!`; `/*` for block |
| TypeScript, JavaScript   | `/** … */` (JSDoc / TSDoc)                        | `/*` opens; ` * ` on continuation    |
| Go                       | `// …` block immediately above the declaration    | `//`                                 |
| Swift                    | `/// …` or `/** … */`                             | `//` or `/*`                          |
| PHP                      | `/** … */` (PHPDoc)                               | `/*` opens; ` * ` on continuation    |
| Ruby                     | `# …` lines (RDoc / YARD)                         | `#` (see note 4.1)                    |
| Python                   | `""" … """` triple-quoted docstring               | special-cased (see note 4.2)         |

This table documents the doc-comment *conventions* for the languages `grund` is built to serve. It is not the gate: the gate is the `[scan] comment_prefixes` list ([§FS-config.3.5](../functional-spec/FS-config.md#35-scan--what-gets-walked)), whose default also recognizes `;` (Lisp / Scheme / Clojure), `--` (SQL / Haskell / Lua / Ada), and bare `*` / `/*` block-comment lines. Any line whose first non-whitespace run is a configured prefix can host a declaration heading or a citation; a language not in the table still works as long as its comment marker is in `comment_prefixes`.

Before declaration, section, or citation detection runs on a source file, the scanner normalizes each eligible comment/docstring line to the content the author meant:

- `//`, `///`, and `//!` line comments strip the full leading comment marker and one following space when present. Therefore `/// AR-001-router: Router`, `//! AR-001-router: Router`, and `// AR-001-router: Router` all expose the same declaration content: `AR-001-router: Router`.
- `#`, `;`, and `--` line comments strip that marker and one following space when present. Therefore Python/Ruby `# AR-001-router: Router` exposes `AR-001-router: Router`; a bare source line `AR-001-router: Router` is not a declaration outside a Python docstring, because it has no comment marker.
- Block comments strip the opener (`/*` or `/**`) and closer (`*/`) when they appear on their own content lines. Continuation lines strip one optional leading `*` plus one following space when present. Therefore ` * AR-001-router: Router` exposes `AR-001-router: Router`.
- Python triple-quoted docstrings in `.py` files enter docstring mode for both `"""` and `'''`. The opening and closing quote-only lines are not content; lines inside are scanned as docstring content. Therefore a class or module docstring containing `AR-001-router: Router` declares `AR-001-router`.
- The normalization is line-local and deterministic. It does not parse the host language beyond recognizing the comment/docstring form above; after normalization, the same heading and citation regexes from §2.1 through §2.3 apply.

The following inline declarations are all required to be recognized under the default scan settings:

```rust
/// AR-001-router: Router
/// Routes requests by path.

//! AR-002-module: Module architecture

/**
 * AR-003-block: Block comment spec
 * ## 1. Contract
 */
```

```go
// AR-004-handler: Handler
// Handles HTTP requests.
```

```python
"""
AR-005-service: Service
## 1. Contract
"""
class Service:
    pass
```

```ruby
# AR-006-job: Job
# Runs background work.
```

A canonical example — a Java class whose Javadoc *is* the architectural spec:

```java
/**
 * AR-event-bus: Asynchronous event distribution
 *
 * ## 1. Responsibilities
 * The event bus owns subscription state and …
 *
 * ## 2. Threading model
 * Single-writer, multi-reader …
 */
public final class EventBus { … }
```

Matched by the matching stub `docs/architecture/AR-<event-bus>.md`:

```
# AR-event-bus: [src/main/java/com/example/EventBus.java](src/main/java/com/example/EventBus.java)
```

### 4.1 Ruby and Python edge cases

- **Ruby** uses `#` as the comment marker. The declaration itself starts after that marker, so the canonical Ruby form is `# AR-<event-bus>`, not a markdown heading inside the comment.
- **Python** docstrings are not comments but string literals (`""" … """`). The scanner has a small docstring mode for `.py`: when a triple-quoted string opens, lines inside it are scanned the same way as comment continuation lines until the matching close. This lets a Python class or module docstring be a fully-featured spec home.

## 5. Why regex, not a parser

Specs live in markdown *and* in source-file doc-comments across half a dozen languages. A real parser per language would be far more code and far slower than a single line-oriented regex pass. The scheme is deliberately designed to be regex-recognizable: the heading shape is unambiguous and the citation shape is anchored on word boundaries.

The trade-off: we cannot reason about the surrounding code structure. We do not need to — IDs are syntactic, not semantic. The link in the stub heading is the only structural pointer between a stub and the code that hosts the inline spec, and it is verified by [§AR-checker.2.4](../../crates/grund-core/src/checker.rs).

The marker character recognized in citations follows [§DF-reference-marker](../decisions/functional/DF-reference-marker.md#df-reference-marker-use--as-the-reference-marker-with--as-the-typing-trigger); the regex shape changes when the marker is reconfigured per [§GOAL-configurable](../goals.md#goal-configurable-every-default-is-overridable).

## 6. E2E case declarations

`E2E` is reserved in the default `[[kinds]]` set, and case-directory discovery follows the directory contract below.

The `E2E` kind is the one kind not declared by a heading line. An `E2E` declaration is a **case directory** directly under the `E2E` kind's `[[kinds]] folder` (default `e2e/cases`). The directory's name is the declared ID with the leading `{kind}` placeholder and its following literal stripped — under the default `[id] format = "{kind}-{number}-{slug}"`, a directory `007-login` declares `E2E-007-login`; under `{kind}-{slug}`, `login` declares `E2E-<login>`; under `{kind}-{number}`, `007` declares `E2E-007`. The directory name must match the format with the kind portion removed; directories that do not (e.g. `.gitkeep`, or a folder with no `expected.exit`) are skipped, so `e2e/cases/` itself never becomes a declaration.

A case directory is recognized as a declaration only if it contains an `expected.exit` file (the minimal marker of a real case). The `Declaration` recorded for it carries the directory path with `line = 1`, an empty section set (the fixture file set is not a numbered-heading tree, so any section-bearing citation of an `E2E` ID — a `.2` suffix and so on — is a missing-section error per [§AR-checker.2.3](../../crates/grund-core/src/checker.rs)), and the deterministic, sorted list of the case's fixture files plus the invocation (`command.args` contents, or the implicit `grund check` when absent) and the expected exit code — this is the "body" [§FS-show.2.4](../functional-spec/FS-show.md#24-e2e-cases) prints. E2E declarations are never stubs, are never hosted in code, and are not reported as unused when no spec cites them.

Citations of an `E2E` ID resolve like any other: an `E2E-<name>` cite from a spec ("proven by …") is a dangling-ref error ([§AR-checker.2.2](../../crates/grund-core/src/checker.rs)) when no `e2e/cases/<name>/` case directory exists.
