# Test Plan: `atproto-devtool test labeler`

Generated against plan `docs/implementation-plans/2026-04-13-test-labeler/`
after final automated review passed (HEAD `2c10045`).

## Prerequisites

- Clean worktree, `cargo build` succeeds.
- `cargo test` passing from a clean run.
- `cargo clippy -- -D warnings` clean.
- `cargo fmt --check` clean.
- Network access to the public internet (bsky.app, plc.directory).
- A POSIX shell; steps assume bash-like redirection.
- Binary path: `cargo run --` or the compiled `target/debug/atproto-devtool`.

## Phase 1: Smoke against a real, known-good labeler (AC1.1, AC6.1)

| Step | Action | Expected |
|------|--------|----------|
| 1.1 | `cargo run -- test labeler moderation.bsky.app` | Exit 0. Output contains `== Identity ==`, `== HTTP ==`, `== Subscription ==`, `== Crypto ==`. Every check line starts with `[OK]` or `[WARN]` (not `[FAIL]`). Footer shows 0 spec violations. |
| 1.2 | `echo $?` immediately after 1.1 | `0`. |
| 1.3 | Re-read the identity section from 1.1 | `target_resolved` reports the did:plc DID for moderation.bsky.app (not a handle). `labeler_endpoint_parseable` shows `https://mod.bsky.app` (or whatever the labeler service endpoint resolves to). |

## Phase 2: Target-form parity (AC1.1 – AC1.4)

| Step | Action | Expected |
|------|--------|----------|
| 2.1 | `cargo run -- test labeler did:plc:ar7c4by46qjdydhdevvrndac` (the moderation.bsky.app DID) | Exit 0. Identity, HTTP, subscription, crypto all run. Output is structurally identical to step 1.1 except the `target` header line shows the DID verbatim. |
| 2.2 | Resolve moderation.bsky.app's labeler endpoint from step 1.1 output (call it `<ENDPOINT>`, e.g. `https://mod.bsky.app`). Run `cargo run -- test labeler <ENDPOINT>` | Exit 0. Identity stage rows are `[SKIP]` with reason mentioning "no DID supplied" (or similar). Crypto stage is `[SKIP]`. HTTP and subscription run normally. |
| 2.3 | Run `cargo run -- test labeler <ENDPOINT> --did did:plc:ar7c4by46qjdydhdevvrndac` | Exit 0. Identity + crypto now run because DID is supplied. An `identity::endpoint_matches_target` (or similar) check passes because endpoints match. |
| 2.4 | Run `cargo run -- test labeler https://api.bsky.app --did did:plc:ar7c4by46qjdydhdevvrndac` (deliberate mismatch) | Exit 1. Identity stage reports a SpecViolation on the endpoint-comparison check; the diagnostic points at the mismatched URL. Footer counts 1 spec violation. |

## Phase 3: Network classification (AC2.8, AC3.6, AC4.6)

| Step | Action | Expected |
|------|--------|----------|
| 3.1 | `cargo run -- test labeler does-not-resolve-at-all.invalid` | Exit 0. Identity stage has `target_resolved` = `[NET]` (NetworkError). Downstream stages blocked with `[SKIP]` + reason. Footer reports 1 network error, 0 spec violations. |
| 3.2 | `cargo run -- test labeler https://127.0.0.1:1/` | Exit 0. Identity + crypto `[SKIP]`, HTTP stage reports `[NET]` on queryLabels reachability. Subscription stage reports `[NET]` on websocket connect. No `[FAIL]` rows. |
| 3.3 | `echo $?` after each of 3.1/3.2 | `0` — network errors do not fail the run. |

## Phase 4: Verbose + no-color rendering (AC6.6, AC6.7)

| Step | Action | Expected |
|------|--------|----------|
| 4.1 | `cargo run -- test labeler --no-color moderation.bsky.app \| cat` | Stdout contains `[OK]`/`[SKIP]`/`[FAIL]` glyphs but NO ANSI escape sequences (`\x1b[` absent). Piping through `cat` proves TTY detection is off; `--no-color` should still suppress color. |
| 4.2 | `NO_COLOR=1 cargo run -- test labeler moderation.bsky.app \| cat` | Same as 4.1. |
| 4.3 | `cargo run -- test labeler --verbose moderation.bsky.app 2>/tmp/stderr.txt >/tmp/stdout.txt` then inspect `/tmp/stderr.txt` | Stderr contains `DEBUG` lines mentioning at least `atproto_devtool::commands::test::labeler::identity`, `::http`, `::subscription`, and `::crypto`. |
| 4.4 | `diff <(cargo run -- test labeler --no-color moderation.bsky.app) <(cargo run -- test labeler --verbose --no-color moderation.bsky.app)` | Stdout is identical between verbose and non-verbose runs (verbose only affects stderr). |

## Phase 5: CLI surface (AC1.5, AC1.6, AC4.7 CLI path, AC6.8)

| Step | Action | Expected |
|------|--------|----------|
| 5.1 | `cargo run -- test labeler --help` | Exit 0. Help text lists `--did`, `--subscribe-timeout`, `--verbose`, `--no-color`, and the positional `<TARGET>` argument. Each flag has a short description. |
| 5.2 | `cargo run -- test labeler` (no target) | Exit code 2. Stderr contains a clap usage error mentioning the missing argument. |
| 5.3 | `cargo run -- test labeler 'not a valid target @#$'` | Exit code 2. Error mentions that the target must be a handle, DID, or https URL. |
| 5.4 | `cargo run -- test labeler --subscribe-timeout 500ms moderation.bsky.app` | Exit code 2. Error contains the substring "at least 1 second". |
| 5.5 | `cargo run -- test labeler --subscribe-timeout 0 moderation.bsky.app` | Exit code 2. Any parse error is acceptable. |

## End-to-End: Real PLC-backed labeler, full pipeline

Purpose: verify that snapshot-backed fake runs correspond to reality for a production labeler.

1. `cargo run -- test labeler moderation.bsky.app >/tmp/real.txt 2>&1`.
2. Compare the check-id rows of `/tmp/real.txt` with the `all_pass_exits_zero_and_renders_all_ok.snap` snapshot (ignoring header lines that encode target-specific values). Every check id that appears in the snapshot should also appear in the real run with status `[OK]`, `[WARN]`, or `[SKIP]`.
3. Verify that timing rows (`elapsed: Nms`) are present and plausible (< 30s total wall-clock).

## End-to-End: Historic key rotation (AC5.2 manual variant)

Purpose: exercise the PLC-history fallback path against a real labeler that has rotated keys.

1. Identify a production did:plc labeler whose PLC audit log shows a key rotation for the `atproto_label` verification method. If none is known, skip this scenario — it is already covered by the fake-backed end-to-end test and no crafted-fixture variant is required.
2. If a candidate exists: `cargo run -- test labeler <handle>`.
3. Expected: exit 0. Crypto stage reports `[OK]` with an advisory listing the historic key id(s) used to verify signed labels.

## Human Verification Required

| Criterion | Why Manual | Steps |
|-----------|------------|-------|
| AC1.1 – AC1.4 real-target parity | Fake-backed snapshots cannot prove the real resolver/PDS path | Phase 1 steps 1.1-1.3 + Phase 2 steps 2.1-2.4 |
| AC2.8 real DNS failure classification | Real DNS resolvers have OS-specific error shapes; only real runs validate the classification | Phase 3 step 3.1 |
| AC3.6 real unreachable-host classification | Only real network failure exercises the reqwest transport-error path end-to-end | Phase 3 step 3.2 |
| AC4.6 real websocket connect failure | Only real network failure exercises tokio-tungstenite's error mapping | Phase 3 step 3.2 (same run covers both) |
| AC5.2 rotated-key advisory | Depends on a real labeler having historic keys in PLC; fake path already covered | Historic-key end-to-end scenario above |
| AC6.7 real DEBUG instrumentation | Confirms the `tracing` wiring actually fires from the real stages, not just that stderr has *some* DEBUG line | Phase 4 step 4.3 |

## Traceability

| Acceptance Criterion | Automated Test | Manual Step |
|----------------------|----------------|-------------|
| AC1.1 | `labeler_identity::handle_resolution_happy_path` + `labeler_endtoend::all_pass...` | Phase 1, 2.1 |
| AC1.2 | `did_plc_direct_happy_path`, `did_web_direct_happy_path` | Phase 2.1 |
| AC1.3 | `endpoint_only_no_did_skips_identity` + `skipped_reasons_rendered` | Phase 2.2 |
| AC1.4 | `endpoint_mismatch_spec_violation` | Phase 2.3, 2.4 |
| AC1.5 | `pipeline::parse_target_unrecognised` + `bootstrap_failure_exits_two` | Phase 5.3 |
| AC1.6 | `help_lists_all_flags` | Phase 5.1 |
| AC2.1 | `healthy_plc_renders_all_ok` | Phase 1.1 |
| AC2.2 | Same fixture as AC2.1 | Phase 1.1 |
| AC2.3 | `missing_service_renders_spec_violation_with_span` | — |
| AC2.4 | `missing_signing_key_renders_spec_violation` | — |
| AC2.5 | `non_https_endpoint_renders_spec_violation` | — |
| AC2.6 | `missing_labeler_record_renders_404_distinct_from_transport` | — |
| AC2.7 | `empty_policies_renders_spec_violation_with_span` | — |
| AC2.8 | `plc_directory_unreachable_renders_network_error` | Phase 3.1 |
| AC3.1 | `http_healthy_renders_all_ok` | Phase 1.1 |
| AC3.2 | `http_healthy_renders_all_ok` (two pages) | — |
| AC3.3 | `http_empty_labeler_emits_advisory` | — |
| AC3.4 | `http_malformed_schema_fails_with_source_span` | — |
| AC3.5 | `http_ignored_cursor_fails` | — |
| AC3.6 | `http_transport_error_renders_network_error` | Phase 3.2 |
| AC4.1 | `backfill_completes_within_budget_passes` | Phase 1.1 |
| AC4.2 | `backfill_exceeds_budget_triggers_live_tail` | — |
| AC4.3 | `empty_stream_advisories` | — |
| AC4.4 | `malformed_frame_emits_spec_violation` | — |
| AC4.5 | `error_frame_malformed_payload_spec_violation` | — |
| AC4.6 | `unreachable_endpoint_network_error`, `live_tail_connect_failure_emits_network_error` | Phase 3.2 |
| AC4.7 | `subscribe_timeout_below_floor_rejected` | Phase 5.4, 5.5 |
| AC5.1 | `all_pass_exits_zero_and_renders_all_ok` | Phase 1.1 |
| AC5.2 | `current_key_fail_history_pass_exits_zero_with_advisory` | Historic-key scenario (best effort) |
| AC5.3 | `empty_labeler_skipped_crypto_only_advisory_exits_zero` | — |
| AC5.4 | `did_web_current_key_fail_exits_one` | — |
| AC5.5 | `current_key_fail_history_also_fail_exits_one` | — |
| AC5.6 | `canonicalization_error_distinct_diagnostic_code` | — |
| AC5.7 | `plc_directory_unreachable_network_error_no_false_fail` | — |
| Canonicalizer golden | `crypto::tests` (moderation.bsky.app + xblock.aendra.dev embedded fixtures) | Phase 1.1 (real labeler sigs verify against PLC keys) |
| AC6.1 | `all_pass_exits_zero_and_renders_all_ok` | Phase 1.1, 1.2 |
| AC6.2 | `identity_only_failure_exits_one_with_severity_breakdown`, `http_decode_failure_exits_one` | Phase 2.4 |
| AC6.3 | `exit_code_summary_for_network_only_run_is_zero`, `subscription_decode_error_exits_one` | Phase 3 (all steps) |
| AC6.4 | `empty_labeler_skipped_crypto_only_advisory_exits_zero` | — |
| AC6.5 | `skipped_reasons_rendered` | Phase 2.2 |
| AC6.6 | `no_color_renders_without_ansi` | Phase 4.1, 4.2 |
| AC6.7 | `verbose_flag_emits_debug_tracing_to_stderr` | Phase 4.3, 4.4 |
| AC6.8 | `bootstrap_failure_exits_two` | Phase 5.2 |
