# Persona reference

A **persona** is a named identity you can impose into images generated by
`plakat scenario`. Each persona is a reference photo plus a few tuning
parameters; a task references a persona by name and routes through the
SD 1.5 portrait pipeline (IP-Adapter-Plus-Face) instead of the scenario's
main t2i or Flux pipeline.

Tasks can use **one** persona over the whole image, or **several**
personas with per-persona bounding boxes that get composited together
via region-masked inpainting.

This document is the full reference for the feature. For an example you
can copy from, see [`examples/scenario.hjson`](../examples/scenario.hjson).
For the standalone CLI (without a scenario), see the **`plakat portrait`**
section in [GENERATE.md](GENERATE.md).

Four identity strategies ship: `plus-face` (SD 1.5), `plus-face-sdxl`
(SDXL), `faceid` (SD 1.5, ArcFace-based), `faceid-sdxl` (SDXL,
ArcFace-based). FaceID strategies additionally need a user-supplied
ArcFace weights file — see [FaceID setup](#faceid-setup). Optional SCRFD
auto-detection wires up via `PLAKAT_SCRFD_WEIGHTS` /
`PLAKAT_SCRFD_HF`. Run `plakat doctor [--verify]` any time to inspect
your environment.

---

## Contents

- [Quick start](#quick-start)
- [HJSON schema](#hjson-schema)
- [Persona fields](#persona-fields)
 - [`name`](#name)
 - [`photo`](#photo)
 - [`identity`](#identity)
 - [`face-strength`](#face-strength)
 - [`face-bbox`](#face-bbox)
 - [`face-landmarks`](#face-landmarks)
 - [`negative`](#negative-on-persona)
- [Task usage: `task.personas`](#task-usage-taskpersonas)
- [The per-image pipeline with personas](#the-per-image-pipeline-with-personas)
- [Output file naming](#output-file-naming)
- [Interaction with other scenario fields](#interaction-with-other-scenario-fields)
- [Photo preparation](#photo-preparation)
- [Tuning identity vs prompt](#tuning-identity-vs-prompt)
- [Workflows](#workflows)
- [Validation and error messages](#validation-and-error-messages)
- [First-run downloads](#first-run-downloads)
- [Memory cost](#memory-cost)
- [Testing a persona standalone](#testing-a-persona-standalone)
- [Troubleshooting](#troubleshooting)
- [Limits](#limits)
- [FaceID setup](#faceid-setup)
- [Optional SCRFD auto-detection](#optional-scrfd-auto-detection)

---

## Quick start

```hjson
{
 enhancer: deepseek
 model: sd15
 out: ./out/scenario

 personas:
 [
 {
 name: alice
 photo: ./refs/alice.jpg
 face-strength: 0.85
 }
 ]

 scene:
 [
 {
 name: cafe
 prompt: "a small Parisian cafe, marble tables, brass fixtures"
 }
 ]

 weather:
 [
 {
 name: morning
 prompt: "soft golden morning light through a side window"
 }
 ]

 tasks:
 [
 {
 name: alice_at_cafe
 scene: cafe
 weather: morning
 prompt: "having an espresso, looking thoughtful"
 personas:
 [
 alice
 ]
 }
 ]
}
```

Run:

```bash
export DEEPSEEK_API_KEY=sk-...
plakat scenario examples/yourfile.hjson
# or validate first without generating:
plakat scenario examples/yourfile.hjson --dry-run
```

Output: `./out/scenario/alice_at_cafe/plakat-portrait-<seed>.png`.

---

## HJSON schema

The full persona-related surface looks like this:

```hjson
{
 # ===== top-level (alongside scene / weather / tasks) =====
 personas:
 [
 {
 name: alice # required, unique
 photo: ./refs/alice.jpg # required; PNG/JPEG/WebP
 identity: plus-face # plus-face | plus-face-sdxl | faceid | faceid-sdxl
 face-strength: 0.85 # default 0.8 (0..2)
 negative: "smiling, mustache" # optional; prepended to task negative
 }
 # … more personas
 ]

 # ===== inside a task: two accepted forms =====
 tasks:
 [
 # --- Form 1: bare-name. One persona fills the whole image.
 {
 name: alice_at_cafe
 scene: cafe
 weather: morning
 prompt: "having an espresso"
 personas:
 [
 alice
 ]
 }

 # --- Form 2: {name, bbox}. One or more personas, each
 # composited into a region. `bbox: [x0, y0, x1, y1]` is normalised
 # to [0, 1] in image space (0,0 = top-left).
 {
 name: alice_and_bob_meeting
 scene: boardroom
 weather: indoor
 prompt: "two colleagues discussing a document"
 personas:
 [
 {
 name: alice
 bbox: [0.0, 0.1, 0.45, 0.9]
 }
 {
 name: bob
 bbox: [0.55, 0.1, 1.0, 0.9]
 }
 ]
 }
 ]
}
```

**Form rules**
- Within one task you must pick one form — bare names OR `{name, bbox}`
 objects. Mixing the two within a single task is rejected at load time.
- Bare form is limited to **exactly 1 persona** (it has no way to place
 multiple personas without bboxes).
- Bbox form accepts 1 or more personas. A single-persona bbox task is
 legal and useful when you want finer control over where the face lands
 than "whole image".

> ⚠️ HJSON inline-list-inside-inline-object (`{ personas: [alice] }` on
> one line) fails to parse with deser-hjson. Always use the multi-line
> form shown above. The same applies to any list field inside a task,
> including bbox lists.

---

## Persona fields

### `name`

**Required. String. Must be unique within the file.**

How tasks refer to this persona (`task.personas: [name]`). Names should be
shell-safe ASCII — `alice`, `bob-young`, `subject_42`. They are matched
case-sensitively.

Validation:

- Duplicate names error at load time: `duplicate persona name "alice"`.
- A task referring to a name that isn't defined errors with the list of
 valid names: `task "..." references unknown persona "ghost" (defined: [alice, bob])`.

### `photo` (single) or `photos` (multi-reference)

**Required. Mutually exclusive — set exactly one.**

#### `photo: <path>`

Single reference photo. Path resolved relative to your **current working
directory** when you run `plakat` — the same convention as `task.style`.
Absolute paths work too.

Supported formats: PNG, JPEG, WebP, BMP, TIFF, anything the `image` crate
opens. Internally rescaled to 224×224 and CLIP-normalised, so the photo's
source resolution doesn't matter much — content does.

#### `photos: [...]` — weighted merge of multiple references

A list of reference photos whose facial features are merged at the
encoder's embedding-space level (not pixel-blended). Use this when:

- You have **multiple photos of the same person** — averaging across them
 dampens single-photo noise (lighting, pose) and produces a more
 representative identity embedding.
- You want to **blend two people** for a fictional likeness or family-
 resemblance look.
- You want to **weight traits across look-alikes** (e.g. 70% one
 reference + 30% another).

Two HJSON forms accepted (mix freely within a list — both parse):

```hjson
# Shorthand strings: "path:weight" (CLI grammar). Weight optional.
photos:
[
 "./refs/alice_smile.jpg:0.5"
 "./refs/alice_neutral.jpg:0.3"
 "./refs/alice_serious.jpg:0.2"
]

# Full-object form: explicit path + weight fields.
photos:
[
 {
 path: ./refs/alice_smile.jpg
 weight: 0.5
 }
 {
 path: ./refs/alice_neutral.jpg
 weight: 0.3
 }
 {
 path: ./refs/alice_serious.jpg
 weight: 0.2
 }
]
```

**Weight semantics**
- Weights are **proportions**, normalized to sum to 1.0 internally. The
 total identity influence is independently controlled by
 `face-strength` (same as for single-photo).
- **Missing weights split the remainder equally.** With three photos and
 weights `[0.6, _, _]`, the unweighted entries each get `(1.0 − 0.6) / 2 = 0.2`.
 With all weights missing, every photo gets `1/N`.
- **All-explicit weights are renormalized** if they don't already sum to
 1.0 — `[7, 3]` is treated the same as `[0.7, 0.3]`.
- **Explicit overflow + auto-weights** errors at load time: if explicit
 weights already exceed 1.0 and at least one entry is auto-weighted,
 there's no room for the auto entries. The error message tells you to
 rebalance or drop the auto entries.
- Negative or NaN weights error at load time.

**How merging works**
Plus-Face strategies (`plus-face`, `plus-face-sdxl`) weighted-sum the
per-photo CLIP-H penultimate hidden states, then run one resampler pass.
FaceID strategies (`faceid`, `faceid-sdxl`) weighted-sum the per-photo
ArcFace 512-d embeddings, **renormalize to unit L2** (ArcFace lives on
the unit sphere), then run one image-projection MLP pass. In both
cases the merge happens *before* the projection module the encoder was
trained on — the projection sees one coherent input, not a stack.

This is fundamentally different from pixel-blending the input photos.
Pixel-blending produces a ghosted/blurry composite that the encoder
reads as a single bad photo; embedding-space merging composes coherent
identity signals from each photo.

**Alignment for multi-photo mode**
- Manual `face-bbox` / `face-landmarks` apply **uniformly to all
 photos**. If you set them with `photos: [...]`, plakat applies the
 same bbox/landmarks to every photo, which is rarely the right
 alignment for multiple photos.
- **SCRFD auto-detection is strongly recommended** for multi-photo
 personas — it detects each photo's face independently. Configure via
 `PLAKAT_SCRFD_WEIGHTS` / `PLAKAT_SCRFD_HF` (see "Optional SCRFD
 auto-detection" below).
- Without SCRFD and without manual alignment, each photo falls back to
 per-photo centre-crop. Acceptable for tight head-shots, mediocre for
 loose photos.

**CLI equivalent:** The `--photo` flag on `plakat portrait` is
repeatable with the same `PATH[:WEIGHT]` grammar:

```bash
plakat portrait "..." \
 --photo alice_smile.jpg:0.5 \
 --photo alice_neutral.jpg:0.3 \
 --photo alice_serious.jpg:0.2
```

Validation: load-time error if any path doesn't exist:
`persona "alice": photo not found at /full/path/to/alice.jpg`.
Setting both `photo:` and `photos:` is rejected:
`persona "alice": photo and photos are mutually exclusive — use one or the other`.

See [Photo preparation](#photo-preparation) for what makes a good
reference photo.

### `identity`

**Optional. String. Default: `plus-face`.**

Which identity-preservation strategy to use for this persona. Every
persona in a scenario must use a strategy that targets the **same model
variant** — `plus-face` (SD 1.5) and `plus-face-sdxl` (SDXL) can't be
mixed in one scenario because the portrait pipeline loads exactly one
base model. Mixing produces a load-time error with the offending kinds
listed.

| Value | Base model | What it does |
|---|---|---|
| `plus-face` (default) | SD 1.5 (cross-attn dim 768) | IP-Adapter-Plus-Face: CLIP-H penultimate hidden state → Perceiver resampler → 16 image tokens → concat onto text tokens. |
| `plus-face-sdxl` | SDXL (cross-attn dim 2048) | Same architecture as `plus-face` but at SDXL's wider cross-attn dim. Reuses the SD 1.5 CLIP-H encoder (the `vit-h` SDXL Plus-Face variant). Visible quality lift on every render — SDXL composes and renders portraits better than SD 1.5. |
| `faceid` | SD 1.5 (cross-attn dim 768) | InsightFace ArcFace embedding (IR-ResNet50, 512-d identity vector) → IP-Adapter-FaceID image-proj → 4 tokens. Markedly better identity preservation than `plus-face` on face-cropped inputs. **Requires `PLAKAT_ARCFACE_WEIGHTS` or `PLAKAT_ARCFACE_HF`** — see [FaceID setup](#faceid-setup). |
| `faceid-sdxl` | SDXL (cross-attn dim 2048) | SDXL variant of `faceid`. Same ArcFace backbone (reuses the same ArcFace env vars), different FaceID image-proj weights (`ip-adapter-faceid_sdxl.bin` from h94). Higher base-quality renders than SD 1.5. Same alignment rules. |
| `instantid` | (planned) | ID embedding + landmarks via a ControlNet-style branch. Best likeness; needs ControlNet integration. Not yet implemented. |

Aliases for `plus-face-sdxl`: `plusface-sdxl`, `plus_face_sdxl`,
`plus-face-xl`, `plusface-xl`, `sdxl-plus-face` — all parse identically.

The scenario derives its portrait base model from this field: if any
persona uses `plus-face-sdxl`, the portrait pipeline loads SDXL.

### `face-strength`

**Optional. Float in `[0.0, 2.0]`. Default: `0.8`.**

Multiplier applied to the persona's image-token contribution before
concatenation onto the text tokens. Maps directly to the diffusers
IP-Adapter `scale` parameter.

| Value | Behaviour |
|---|---|
| `0.0` | Image tokens zeroed out. Equivalent to running the task without a persona at all — pure text-driven. |
| `0.3` | Faint hint of the reference. Useful when you want the prompt to dominate and identity is decorative. |
| `0.5` | Light identity influence. Recognisable hint of the person; prompt steers strongly. |
| `0.8` (default) | Strong likeness, prompt still has firm control. The recommended starting point. |
| `1.0` | Full reference influence. Prompt fights harder for things like pose / wardrobe / lighting. |
| `>1.0` | Over-amplified. Sometimes useful for difficult photos (poor lighting, oblique angle) but the prompt loses control of skin tone, expression, and lighting choices. |

Values are clamped to `[0.0, 2.0]` at runtime. Negative numbers and `NaN`
are not accepted by HJSON's float parser.

### `face-bbox`

**Optional. `[x0, y0, x1, y1]` normalised to `[0, 1]`. Default: `None`.**

Locates the subject's face in the reference photo, so FaceID strategies
can crop to the face before ArcFace embedding. Coordinates are
normalised to the photo's dimensions with origin top-left.

```hjson
{
 name: alice
 photo: ./refs/alice.jpg
 identity: faceid
 face-bbox: [0.20, 0.05, 0.80, 0.65] # face occupies most of the upper half
}
```

CLI equivalent: `--face-bbox 0.20,0.05,0.80,0.65`.

**Who reads it**
| Strategy | Uses `face-bbox`? |
|---|---|
| `faceid`, `faceid-sdxl` | **Yes** — crops to bbox, then resizes to 112×112 for ArcFace. Useful when the face is off-centre, or to disambiguate which face to use in a multi-face photo. |
| `plus-face`, `plus-face-sdxl` | No — CLIP-H processes the whole image regardless, so cropping would just throw away useful context. The field is silently ignored. |

**Why specify a bbox at all (FaceID-only):** ArcFace was trained on
landmark-aligned 112×112 face crops. The default centre-crop only
works when the face is already roughly centred. A wide shot, group
photo, or off-centre framing produces a bad input because the "face"
the centre-crop produces is actually background or a body part.

Validation: bbox bounds, `x0 < x1`, `y0 < y1`, all in `[0, 1]`. Bad
bboxes fail at scenario load (or at CLI argument parse for
`--face-bbox`) with a clear message.

When SCRFD auto-detection is enabled, the detector fills landmarks
automatically — `face-bbox` remains useful for "use *this* face when
there are several in the photo" by manually narrowing the region.

### `face-landmarks`

**Optional. `[[x, y]; 5]` normalised to `[0, 1]`. Default: `None`.**

5-point face landmarks, ordered exactly as `LANDMARK_ORDER` (the
InsightFace convention): `left_eye, right_eye, nose, left_mouth_corner,
right_mouth_corner`. Coordinates are normalised against the photo's
pixel dimensions with origin top-left.

```hjson
{
 name: alice
 photo: ./refs/alice.jpg
 identity: faceid
 face-landmarks:
 [
 [0.40, 0.40] # left eye
 [0.60, 0.40] # right eye
 [0.50, 0.55] # nose
 [0.42, 0.68] # left mouth corner
 [0.58, 0.68] # right mouth corner
 ]
}
```

CLI equivalent: `--face-landmarks 0.40,0.40,0.60,0.40,0.50,0.55,0.42,0.68,0.58,0.68`.

**Who reads it**
| Strategy | Uses `face-landmarks`? |
|---|---|
| `faceid`, `faceid-sdxl` | **Yes** — runs Umeyama's similarity transform to ArcFace's canonical 112×112 template before embedding. **The proper alignment**; closest we get to reference quality. |
| `plus-face`, `plus-face-sdxl` | No — CLIP-H processes the whole image, alignment doesn't apply. Silently ignored. |

**Precedence over `face-bbox`:** when both are set, landmarks win.
Landmarks carry strictly more information (location + rotation + scale)
so the alignment they produce supersedes a bbox crop.

Validation: every component must be in `[0, 1]`. Errors at scenario load
or CLI parse with a specific component index in the message.

SCRFD auto-detection (when enabled) fills these landmarks from any
photo. The field stays useful for "use exactly these landmarks even
if a detector would pick different ones" — handy when a photo has
multiple faces and you want a specific one.

### `negative` (on persona)

**Optional. String. Default: empty.**

A negative prompt that travels with the persona, not the task. Example use:

```hjson
{
 name: alice
 photo: ./refs/alice.jpg
 negative: "smiling, mustache, sunglasses, beard"
}
```

When this persona is imposed onto a task, the persona's negative is
**prepended** to the effective task negative (task-level override > scenario-level
negative > empty), comma-joined:

```
persona.negative + ", " + task.negative
```

Empty persona-negative passes through cleanly (no double comma). Empty
task-negative is also handled. The result is fed to the portrait pipeline.

Why prepended, not replaced: the persona's negatives describe **the
identity** ("alice doesn't wear glasses"), while the task's negatives
describe **the rendering** ("low quality, blurry"). Both should apply.
Prepending also means that if the scenario has a baseline negative
("blurry, deformed"), it still takes effect when this persona is used.

---

## Task usage: `task.personas`

### Form 1 — bare name (single persona, whole image)

```hjson
{
 name: alice_at_cafe
 scene: cafe
 weather: morning
 prompt: "having an espresso"
 personas:
 [
 alice
 ]
}
```

Used when a single persona fills the frame — portraits, character
studies, single-subject scenes. The portrait pipeline runs with the
persona's photo and tokens applied across the entire image, just like
`plakat portrait --photo alice.jpg`.

### Form 2 — `{name, bbox}` (multi-persona, region-masked compositing)

```hjson
{
 name: pair_at_table
 scene: bistro
 weather: golden_hour
 prompt: "two friends sharing dessert"
 personas:
 [
 {
 name: alice
 bbox: [0.0, 0.05, 0.5, 0.95]
 }
 {
 name: bob
 bbox: [0.5, 0.05, 1.0, 0.95]
 }
 ]
}
```

Used when several persona identities should land in distinct image
regions. Each persona supplies a `bbox` — `[x0, y0, x1, y1]` normalised
to `[0, 1]` with `(0, 0)` at the top-left of the image — telling the
pipeline where their face should go.

#### How bbox compositing works

1. A **text-only base image** is generated by the portrait pipeline
 (no photo). This sets up the scene, lighting, and composition for
 the prompt.
2. For each persona, an **inpaint pass** runs: the masked region (the
 persona's bbox) is re-denoised with that persona's image tokens
 applied; the rest of the image stays pinned to the previous step's
 output (the base, or the result of the previous persona's pass).
3. The final composite is decoded and written as
 `plakat-portrait-<seed>.png`.

The technique is RePaint-style latent blending: at each denoise step
inside the loop, the unmasked region is replaced with a freshly re-noised
copy of the base latents, so the denoiser sees a coherent neighbour and
the masked region transitions cleanly into the surrounding image.

#### Bbox placement guidance

- `[0.0, 0.05, 0.5, 0.95]` and `[0.5, 0.05, 1.0, 0.95]` — classic
 two-person side-by-side. Leave ~5% padding top/bottom so the prompt
 can render hands, props, surfaces.
- `[0.0, 0.0, 0.33, 1.0]` / `[0.33, 0.0, 0.67, 1.0]` / `[0.67, 0.0, 1.0, 1.0]`
 — three-person row.
- `[0.25, 0.1, 0.75, 0.7]` — single centred persona with breathing room
 for body / background. Same effect as bare-name form but more control.
- Overlapping bboxes are legal but the **later persona wins** inside the
 overlap (each inpaint pass treats the previous result as its base).
 Use this intentionally for layered compositions; avoid it accidentally.
- The validator requires `x0 < x1`, `y0 < y1`, and all four in
 `[0.0, 1.0]`. Inverted or out-of-range bboxes fail at load time.

#### Form-mixing rejection

Mixing the two forms within a single task errors at load:

```
task "broken" cannot mix bare-name form (`[alice]`) with bbox form
(`[{name:alice, bbox:[...]}]`) in the same task. Pick one. Use bbox
for multi-persona compositing; use bare-name when the persona occupies
the whole image.
```

Bare form is also restricted to exactly 1 entry — multi-persona requires
bboxes:

```
task "boardroom" requests 3 personas (alice, bob, carol) in bare-name
form. Multi-persona requires bboxes — convert to
`[{name:..., bbox:[x0,y0,x1,y1]}, ...]` form.
```

A task without a `personas` field, or with an empty list, runs through
the scenario's regular pipeline (t2i or Flux) — unchanged behaviour.

---

## The per-image pipeline with personas

For each image a task generates:

1. **Generate**:
 - No persona → existing t2i/Flux pipeline →
 `plakat-<seed>.png` (or `plakat-flux-<seed>.png` for Flux).
 - **Form-1 single persona** → portrait pipeline, one denoise pass,
 persona tokens applied across the whole image →
 `plakat-portrait-<seed>.png`.
 - **Form-2 multi-persona (bboxes)** → portrait pipeline, multi-pass
 compositing (described below) → `plakat-portrait-<seed>.png`.
2. **Stylize** (if `task.style` is set) → `<base>-styled.png`. Runs on
 whatever the previous step produced; portrait or not.
3. **Upscale** (if `upscale.upscale: true`) → `<base>-styled-upscaled.png`
 if step 2 ran successfully, else `<base>-upscaled.png`.

### Single-persona pipeline (Form 1)

1. Resolves the persona's photo, CLIP-H-encodes it, runs the Perceiver
 resampler → **16 image tokens** of dimension 768 (SD 1.5 cross-attn).
2. Scales the image tokens by `face-strength`.
3. Concatenates onto the text-token sequence:
 - **Cond** branch: `[text_cond (77) | image_tokens (16)]` →
 `(1, 93, 768)`.
 - **Uncond** branch (when CFG): `[text_uncond (77) | zeros (16)]`.
4. Runs the standard SD 1.5 denoise loop from pure noise with classifier-free
 guidance.
5. Optionally a same-model `--refine` polish pass on the final latents.
6. VAE decode + PNG save.

### Multi-persona compositing pipeline (Form 2)

1. **Base pass:** generate a text-only base from the prompt (no persona).
 Same denoise loop as Form 1 but with `face_strength = 0` — establishes
 the scene, lighting, composition.
2. **Per-persona inpaint passes** (one per persona in declaration order):
 - Build a mask from the persona's bbox at latent resolution
 (`width / 8 × height / 8`). Mask is `1.0` inside the bbox, `0.0`
 outside.
 - CLIP-H-encode the persona's photo → 16 image tokens (scaled by
 `face-strength`), concat onto the text tokens for this pass.
 - Run **RePaint-style latent blending**: at each timestep `t`,
 - re-noise the previous-pass latents at `t` and place them in the
 unmasked region (preserves what's outside the bbox);
 - run a standard denoise step on the combined latents (UNet drives
 the masked region toward the persona's identity);
 - at the final step, replace the unmasked region with the **clean**
 previous-pass latents so the seam pins exactly.
 - Output: a new set of latents with this persona's bbox filled in.
3. **Decode + save** the final composite.

The chaining is deliberate: pass 2 treats the result of pass 1 as its
base, so personas already composited in don't get re-noised by later
passes (except where bboxes overlap, in which case the later persona
wins inside the overlap).

### LoRA interaction

LoRAs from `scenario.loras` are merged into both the UNet and the CLIP-L
text encoder of the portrait pipeline at load time — exactly the same
flow as the t2i pipeline. The merged UNet is reused across **all**
persona passes (base + per-persona inpaints) within a multi-persona
task.

---

## Output file naming

| Task uses... | Output file |
|---|---|
| No persona, SD/SDXL `model` | `<task_dir>/plakat-<seed>.png` |
| No persona, Flux `model` | `<task_dir>/plakat-flux-<seed>.png` |
| One persona | `<task_dir>/plakat-portrait-<seed>.png` |

The downstream style and upscale passes pick the right base file by
reading the task's effective prefix, so:

| With persona + style | `plakat-portrait-<seed>.png`, `plakat-portrait-<seed>-styled.png` |
| With persona + style + upscale | … + `plakat-portrait-<seed>-styled-upscaled.png` |
| With persona + upscale only | `plakat-portrait-<seed>.png`, `plakat-portrait-<seed>-upscaled.png` |

Seeds advance contiguously per the **global** `count`, regardless of
whether a task uses a persona — so re-running a scenario yields the
same composition for the same task name.

---

## Interaction with other scenario fields

### Global fields the portrait pipeline honours

| Field | Behaviour |
|---|---|
| `loras` / `lora-scale` | Merged into the portrait UNet + CLIP-L text encoder, same as the main pipeline. |
| `scheduler` | Used for the portrait denoise loop. Default is whatever the scenario specifies. |
| `refine` / `refine-strength` | Same-model polish runs on the portrait output too. |
| `prompt-header` / `prompt-footer` / `lora-header` / `lora-footer` | Wrapped around the enhanced prompt before it goes to the portrait pipeline. |
| `negative` | Used as the task negative unless overridden per-task. Then persona negative is prepended. |
| `seed` | Same per-task seed plumbing as t2i — the global counter still advances by global `count` per task. |
| `out` | Output root. Per-task subdirectory named after `task.name`. |

### Global fields the portrait pipeline **ignores**

| Field | Why |
|---|---|
| `model` | Portrait is always SD 1.5 (Plus-Face is SD 1.5 only). If you set `model: sdxl`, persona tasks still use SD 1.5; non-persona tasks use SDXL. |
| `refiner` / `refiner-frac` | SDXL refiner doesn't apply to SD 1.5. Silently ignored for persona tasks. |

### Per-task overrides

A persona task obeys per-task overrides the same way a regular task does:

| Override | Applies to persona tasks? |
|---|---|
| `size` / `aspect` | Yes — the portrait pipeline uses the override dimensions. |
| `count` | Yes. |
| `steps` | Yes. |
| `guidance` | Yes. |
| `seed` | Yes. |
| `negative` | Yes; persona-negative is then prepended to this. |
| `scheduler` | Yes. |
| `refine` / `refine-strength` | Yes. |
| `style` / `style-strength` | Yes — stylize runs on the portrait output. |
| `refiner-frac` | Ignored (SD 1.5 portrait has no refiner). |

### Stylize and upscale on persona outputs

Both work normally. The stylize pipeline takes the portrait PNG as its
input image and runs IP-Adapter against `task.style` as the reference;
upscale takes whatever the latest sibling file is (styled if styled,
otherwise the persona output).

**Caveat:** stylize tends to wash out fine facial detail at higher
strengths. With personas, prefer `style-strength: 0.4-0.55` so the
identity holds.

---

## Photo preparation

The reference photo is the single biggest quality lever for `plus-face`
strategies — no automatic face detection or alignment is applied, so
the input is treated as-is.

**What works well**
- Tight head-and-shoulders crop. Face occupies roughly 30–70% of the
 frame's height.
- Neutral or front-facing pose (off-axis up to ~30° is fine).
- Even lighting. Soft daylight or studio light. Hard shadows across the
 face leak into generated outputs as "moody lighting" the prompt can't
 fully wash out.
- Clean background (or at least one with no other faces). The model
 doesn't know which face is the subject.
- Resolution: 512×512 or higher on the face region. Source is rescaled
 to 224×224 internally, but oversampling helps CLIP-H pick out features
 cleanly.
- Neutral expression. Strong expressions (grin, surprise) bleed into
 every generated frame.

**What hurts**
- Wide shots where the face is small.
- Group photos. The model averages identities or grabs the wrong one.
- Heavy occlusion (sunglasses, surgical mask, hat over forehead, hand
 near face).
- Profile shots. Front-facing photos generalise better.
- Strong filters / heavy beauty retouching on the reference. The model
 picks up the filter's artefacts as part of the identity.
- Very low resolution. Below ~150 px on the face, results degrade.
- Compression artefacts. Heavily-compressed JPEGs reduce CLIP-H's ability
 to encode subtle features.

When a persona produces poor results: re-check the photo first before
tweaking `face-strength`.

---

## Tuning identity vs prompt

When the **prompt fights the identity** (the rendered face is generic
or composite-looking despite a clean photo):

1. Raise `face-strength` toward 1.0.
2. Lower `guidance` slightly (`6.0`-`7.0` instead of `7.5+`).
3. Try `--scheduler euler-a` if you're using DDIM — Euler-A often holds
 identity better at low-to-medium step counts.

When the **identity overwhelms the prompt** (the model ignores wardrobe,
pose, or lighting requests in favour of duplicating the photo's vibe):

1. Lower `face-strength` to `0.5-0.7`.
2. Raise `guidance` toward `7.5-9.0`.
3. Use a strong, specific `prompt-header` or `task.prompt` for the
 contested element (e.g. "in a red wool coat, three-quarter angle").

When **the result barely looks like the person** at `face-strength: 1.0`:

The most common cause is the reference photo itself. See
[Photo preparation](#photo-preparation). `plus-face` strategies have a
hard quality ceiling — ~50–70% of the diffusers reference — because
candle 0.8's UNet doesn't expose attention hooks for IP-Adapter's
*decoupled* cross-attention path. If you've optimised the photo and
pushed `face-strength` to 1.5 and likeness is still wrong, switch to a
`faceid` strategy (which uses InsightFace ArcFace embeddings instead of
CLIP-H features) — that's the path forward, not further parameter tuning.

---

## Workflows

### One persona, many scenes

The intended primary use. Define alice once, reference her across many
tasks. The portrait pipeline loads exactly once at scenario start and is
reused.

```hjson
personas:
[
 {
 name: alice
 photo: ./refs/alice.jpg
 face-strength: 0.85
 }
]

tasks:
[
 { name: alice_cafe, scene: cafe, weather: morning, prompt: "espresso", personas: [alice] }
 { name: alice_forest, scene: forest, weather: golden_hour, prompt: "walking", personas: [alice] }
 { name: alice_harbour, scene: harbour, weather: rain, prompt: "lantern", personas: [alice] }
]
```

(Inline objects shown for brevity; real HJSON files need multi-line form.)

### Compare strength variants ("alice-strong" vs "alice-soft")

There's no per-task override of persona params — a task references a
persona by name and uses that persona's settings as-is. The recommended
pattern for A/B comparisons is to define separate personas for the
variants you want to compare:

```hjson
personas:
[
 {
 name: alice_soft
 photo: ./refs/alice.jpg
 face-strength: 0.5
 }
 {
 name: alice_strong
 photo: ./refs/alice.jpg
 face-strength: 1.0
 }
]

tasks:
[
 { name: a1, scene: s, weather: w, prompt: "p", personas: [alice_soft] }
 { name: a2, scene: s, weather: w, prompt: "p", personas: [alice_strong] }
]
```

Re-using the same `seed` across two such tasks isolates the strength
variable — same noise seed, same prompt, different conditioning.

### Persona + style pass

A persona task can still have a `style` reference. The stylize step runs
on the portrait output:

```hjson
{
 name: alice_watercolor
 scene: cafe
 weather: morning
 prompt: "thoughtful"
 personas:
 [
 alice
 ]
 style: ./refs/watercolor-painting.jpg
 style-strength: 0.45
}
```

Recommended `style-strength`: **0.4–0.55**. Higher strengths wash out
fine facial detail and the likeness blurs.

### Persona + ML upscale

```hjson
upscale:
{
 upscale: true
 method: real-esrgan-x4
}
```

The upscale pass picks up `plakat-portrait-<seed>.png` (or
`plakat-portrait-<seed>-styled.png` if styled) and produces
`plakat-portrait-<seed>-upscaled.png` (or `…-styled-upscaled.png`). Real-ESRGAN
preserves facial detail well; this combo is the recommended path for
"final-quality" portrait renders.

### Mixed scenario (some tasks with personas, some without)

Fully supported. The portrait pipeline loads only because at least one
task uses a persona; non-persona tasks continue to use the scenario's
main `model`. The downside is memory cost — see [Memory cost](#memory-cost).

### Two-person scene with distinct identities (Form 2)

```hjson
personas:
[
 {
 name: alice
 photo: ./refs/alice.jpg
 face-strength: 0.85
 }
 {
 name: bob
 photo: ./refs/bob.jpg
 face-strength: 0.85
 }
]

tasks:
[
 {
 name: meeting
 scene: boardroom
 weather: indoor
 prompt: "two colleagues reviewing a printed document, three-quarter angle"
 personas:
 [
 {
 name: alice
 bbox: [0.05, 0.1, 0.48, 0.95]
 }
 {
 name: bob
 bbox: [0.52, 0.1, 0.95, 0.95]
 }
 ]
 }
]
```

Tips:

- **Use wider aspect ratios for multi-persona** (16:9 or 3:2 rather than
 1:1) — gives each bbox enough room for a recognisable face.
- **Leave a small gap between bboxes** (e.g. `0.48` / `0.52` above) to
 avoid the inpaint passes fighting over the same latents.
- **5–10% padding from the edges** lets the prompt render arms, hands,
 surfaces — without padding the bbox edges become hard image edges.
- **First persona becomes the "anchor"**: their pass runs against a
 text-only base, so their bbox gets the cleanest result. Later personas
 composite onto the partially-composited image. For 3+ personas,
 declare the most-important identity first.

### Single-persona with explicit placement (Form 2)

Useful when bare-form's "whole image" treatment puts the face too
high / low / off-centre for your composition:

```hjson
{
 name: alice_at_window
 scene: kitchen
 weather: morning
 prompt: "looking out the window, holding a coffee mug"
 personas:
 [
 {
 name: alice
 bbox: [0.55, 0.2, 0.95, 0.85]
 }
 ]
}
```

Form 2 with a single persona costs the same as a Form-1 task (one base
pass + one inpaint pass — actually slightly more wall-time than Form 1
because Form 1 runs a single combined pass).

---

## Validation and error messages

All persona validation runs at scenario load, before any model
downloads. Each of these is a fast-fail.

| Condition | Error |
|---|---|
| Duplicate persona name | `duplicate persona name "alice"` |
| Persona photo path doesn't exist | `persona "alice": photo not found at /full/path/to/alice.jpg` |
| Persona `identity` unknown | `persona "alice" identity: unknown identity kind "instantid" (try: plus-face, plus-face-sdxl, faceid, faceid-sdxl). InstantID is roadmap — not yet implemented.` |
| Scenario mixes identity strategies | `scenario mixes identity strategies across personas: PlusFaceSdxl (bob); PlusFace (alice). Pick one strategy per scenario — every persona must share the same model variant (all SD 1.5 \`plus-face\`, or all SDXL \`plus-face-sdxl\`).` |
| Task refers to unknown persona name | `task "scene1" references unknown persona "ghost" (defined: [alice, bob])` |
| Task mixes bare-name and bbox forms | `task "t": cannot mix bare-name form ("[alice]") with bbox form ("[{name:alice, bbox:[...]}]") in the same task. Pick one. Use bbox for multi-persona compositing; use bare-name when the persona occupies the whole image.` |
| Bare form with `>1` persona | `task "bare2" requests 2 personas (alice, bob) in bare-name form. Multi-persona requires bboxes — convert to "[{name:..., bbox:[x0,y0,x1,y1]}, ...]" form.` |
| bbox out of range or inverted | `task "bad": persona "alice" bbox [0.5, 0.5, 0.3, 0.7] is invalid (must be [x0,y0,x1,y1] with 0 ≤ x0 < x1 ≤ 1 and 0 ≤ y0 < y1 ≤ 1)` |
| Standalone CLI: model / identity mismatch | `identity strategy PlusFaceSdxl targets cross_attn_dim 2048 but model "sd15" (Sd15) expects 768. Pick an identity that matches the model: SD 1.5 → \`plus-face\`, SDXL → \`plus-face-sdxl\`.` |

These also surface in `--dry-run`, so you can iterate on the schema
without paying any download or generation cost.

In `--dry-run`, each persona task additionally prints a line per persona
summarising what would happen:

```
 (dry-run) would impose persona "alice" via portrait pipeline (photo /refs/alice.jpg, strength 0.85, ok)
 (dry-run) would impose persona "alice" via portrait pipeline (photo /refs/alice.jpg, strength 0.85 bbox=[0.00,0.10,0.45,0.90], ok)
 (dry-run) would impose persona "bob" via portrait pipeline (photo /refs/bob.jpg, strength 0.80 bbox=[0.55,0.10,1.00,0.90], ok)
```

The trailing `ok` or `MISSING` reflects a final-recheck against the
filesystem — if the photo got deleted between load and the per-task
log, you'd see `MISSING`. Form-2 tasks log one line per persona pass.

---

## First-run downloads

Persona tasks trigger downloads beyond the scenario's base model. The
exact set depends on the persona's `identity` strategy.

### `identity: plus-face` (SD 1.5)

| Weight | Size | Source | Notes |
|---|---|---|---|
| SD 1.5 base (UNet + VAE + CLIP-L) | ~4 GB | `stable-diffusion-v1-5/stable-diffusion-v1-5` | Shared with `plakat generate --model sd15`, `plakat stylize`, `plakat portrait`. Downloaded once. |
| CLIP-H image encoder | ~2.5 GB | `h94/IP-Adapter/models/image_encoder/` | Shared with `plakat stylize` and `plus-face-sdxl`. Downloaded once. |
| IP-Adapter-Plus-Face weights | ~50 MB | `h94/IP-Adapter/models/ip-adapter-plus-face_sd15.safetensors` | Persona-specific. |

Total first-run cost: **~6.5 GB**.

### `identity: plus-face-sdxl` (SDXL)

| Weight | Size | Source | Notes |
|---|---|---|---|
| SDXL base (UNet + VAE + dual CLIP) | ~7 GB | `stabilityai/stable-diffusion-xl-base-1.0` | Shared with `plakat generate --model sdxl`. Downloaded once. |
| CLIP-H image encoder | ~2.5 GB | `h94/IP-Adapter/models/image_encoder/` | Same file as `plus-face` (the `vit-h` SDXL Plus-Face reuses it). |
| IP-Adapter-Plus-Face SDXL weights | ~50 MB | `h94/IP-Adapter/sdxl_models/ip-adapter-plus-face_sdxl_vit-h.safetensors` | Persona-specific. |

Total first-run cost: **~9.5 GB**. The shared CLIP-H means switching
between SD 1.5 and SDXL personas across scenarios only re-downloads the
strategy-specific (~50 MB) safetensors.

All weights live in `~/.cache/huggingface/hub` (or your `--cache-dir`
override). Subsequent runs cost nothing on the download path.

---

## Memory cost

The portrait pipeline is loaded **in addition to** any other pipelines
the scenario needs. A worst-case scenario that uses Flux as its main
model AND stylize AND personas loads:

- Flux transformer + AE + T5 + CLIP-L (~30 GB on first run; ~17 GB resident)
- Stylize pipeline (SD 1.5 + CLIP-H image encoder) — ~5 GB resident
- Portrait pipeline (SD 1.5 + CLIP-H image encoder) — ~5 GB resident

CLIP-H is loaded twice — once for stylize, once for portrait — even
though it's the same weights file on disk. Sharing it across pipelines
is a future optimisation; today the duplication is tolerated.

If memory is tight: stick to SD 1.5 as your scenario `model`. The
portrait pipeline then sits next to a single ~5 GB t2i pipeline, total
~10 GB.

For Apple Silicon chip + memory tiers and unified-memory headroom
guidance, see [`APPLE_REQUIREMENTS.md`](APPLE_REQUIREMENTS.md).

---

## Testing a persona standalone

Before stuffing a new persona into a scenario, sanity-check the photo
+ strength combo with `plakat portrait`:

```bash
# SD 1.5
plakat portrait "studio portrait, neutral expression, soft lighting" \
 --photo ./refs/alice.jpg \
 --face-strength 0.85 \
 --size 768x1024 \
 --steps 30 \
 --seed 42 \
 --out ./out/persona-test

# SDXL — pair --model sdxl with --identity plus-face-sdxl. Use a wider
# size (1024×1024 or 1024×1366) to let SDXL render the full detail it's
# trained for.
plakat portrait "studio portrait, neutral expression, soft lighting" \
 --photo ./refs/alice.jpg \
 --model sdxl \
 --identity plus-face-sdxl \
 --face-strength 0.85 \
 --size 1024x1366 \
 --steps 30 \
 --seed 42 \
 --out ./out/persona-test
```

If the result looks like the person at one of `0.6 / 0.8 / 1.0 / 1.2`,
copy that value into the persona's `face-strength`. If none of those
work, the photo is the bottleneck — see [Photo preparation](#photo-preparation).

`--model sdxl` paired with `--identity plus-face` (or `--model sd15` with
`--identity plus-face-sdxl`) errors at load with a clear cross-attn-dim
mismatch message, so you can't accidentally combine wrong pairs.

The standalone `portrait` CLI is documented in detail in
[GENERATE.md](GENERATE.md#plakat-portrait).

---

## Troubleshooting

**"The face looks like the right person but is squished / stretched."**

Aspect ratio mismatch. SD 1.5 prefers near-square or 3:4 portrait
ratios; very wide ratios produce face distortion. Use `--size 768x1024`
or `--aspect 3:4` per task, or set it scenario-wide.

**"Two people show up in the output even though I only specified one persona."**

A combination of "face" in the prompt + a reference photo can trigger
SD 1.5's tendency to render two figures. Try negative-prompting: add
`"two people, multiple faces, group photo"` to the persona's `negative`
or the task's `negative`.

**"The face is right but pose/clothing follows the reference photo too much."**

Lower `face-strength` to 0.5–0.7. The reference photo's pose and
clothing are encoded into the CLIP-H tokens; stronger conditioning
carries those forward. Prompt the wardrobe/pose explicitly.

**"The image looks AI-generated even with a real reference."**

This is the `plus-face` quality ceiling. Without decoupled cross-
attention (candle 0.8's UNet doesn't expose the hooks), identity tokens
compete with text tokens for the same attention budget. For more
photographic likeness, switch to a `faceid` strategy (uses ArcFace
embeddings, ships a UNet LoRA that lifts identity preservation
substantially). InstantID would push further but isn't implemented yet.

Workarounds without switching strategies:
- Use a realistic-portrait LoRA (e.g. `*Realistic*` SD 1.5 LoRA) at
 `lora-scale: 0.7`. The LoRA shapes the rendering style; the persona
 shapes the identity. They stack additively.
- Add a Real-ESRGAN upscale pass (`real-esrgan-x4`). Fine-detail recovery
 often pushes the face closer to "photo-like".
- Add a same-model polish (`refine: 6, refine-strength: 0.3`). Sharpens
 details and removes some AI artefacts.

**"`Error: parsing HJSON ... UnexpectedChar at N:M`"**

Inline-list-inside-inline-object isn't supported by deser-hjson. Always
write `task.personas` as:

```hjson
personas:
[
 alice
]
```

not `personas: [alice]` on a single line. Same constraint applies to
`loras: [...]` and any other list inside a task.

**"Persona task ran but the output looks like text-only generate."**

Possible causes:
- The photo path doesn't resolve from the **current working directory**
 where you ran `plakat`. Use absolute paths to verify, then convert
 back to relative paths once it works.
- `face-strength` was accidentally set to 0.0 (or very low).
- The `task.personas` list is empty (`[]`) — this counts as "no
 persona". Needs at least one entry (bare name or `{name, bbox}`) to
 engage the portrait pipeline.

---

## Limits

Honest, documented constraints — not bugs:

1. **One base model per scenario.** Within one scenario, every persona's
 `identity` must target the same model variant (all SD 1.5 or all SDXL).
 Mixing is rejected at load time because the portrait pipeline loads
 exactly one base model.
2. **No automatic face crop on `plus-face*` strategies.** Curate the
 reference photo or use `--face-bbox`. (FaceID strategies *can* use
 optional SCRFD auto-detection — see [Optional SCRFD auto-detection](#optional-scrfd-auto-detection).)
3. **No per-task override of persona parameters.** Define a second persona
 if you need a strength variant (e.g. `alice-soft` vs `alice-strong`).
4. **No persona-level LoRAs.** Use scenario-level `loras`. The merged
 UNet is reused across all persona passes within a task.
5. **Identity quality ceiling for `plus-face*` strategies: ~50–70% of
 diffusers reference.** candle 0.8's UNet exposes no cross-attention
 hooks, so the *decoupled* IP-Adapter path (separate `to_k_ip` /
 `to_v_ip` per block) is not wired up — identity tokens travel via
 the same cross-attention as text. `faceid` / `faceid-sdxl` strategies
 bypass this ceiling by using ArcFace embeddings + an automatically-
 applied UNet LoRA from h94, landing closer to ~80–90% of reference.
6. **SDXL micro-conditioning is unused.** SDXL's `add_embedding` (pooled
 CLIP-G + size/crop time-ids) is not wired up because candle 0.8's UNet
 doesn't expose the projection. Same gap as the base SDXL t2i path —
 the model loads and runs but doesn't benefit from size conditioning.
7. **Multi-persona compositing has visible bbox seams** at high contrast.
 The RePaint-style blend hides most transitions, but a face inpainted
 onto a strongly-coloured background can show a faint rectangular
 boundary. Workarounds: pad the bbox slightly so the face sits inside
 it; pick a prompt where each persona's neighbourhood is visually
 similar (same lighting, same surface).
8. **Multi-persona wall time scales linearly with persona count.** A
 2-persona task runs 3 denoise loops (base + 2 inpaints); a 3-persona
 task runs 4. Each loop is full `--steps` long.
9. **InstantID is not implemented.** Listed in the `identity` schema as
 a placeholder; the underlying ControlNet integration isn't ported yet.

---

## FaceID setup

The `faceid` and `faceid-sdxl` strategies need two weight sources:

1. **IP-Adapter-FaceID image-proj + UNet LoRA** — auto-downloaded from
 `h94/IP-Adapter-FaceID` on first use. Nothing for you to do.
2. **ArcFace IR-ResNet50** — supply via one of two routes below.

### Route A — local file (`PLAKAT_ARCFACE_WEIGHTS`)

```bash
# 1. Download the antelopev2 bundle from InsightFace:
curl -L -o antelopev2.zip \
 https://github.com/deepinsight/insightface/releases/download//antelopev2.zip
unzip antelopev2.zip # → antelopev2/

# 2. Convert the ONNX to safetensors (one-time, ~250 MB output):
python -c "import onnx, torch
from onnx2torch import convert
from safetensors.torch import save_file
m = convert(onnx.load('antelopev2/w600k_r50.onnx'))
save_file(m.state_dict(), 'arcface_r50.safetensors')"

# 3. Point plakat at it:
export PLAKAT_ARCFACE_WEIGHTS=$(pwd)/arcface_r50.safetensors
```

### Route B — HuggingFace-hosted (`PLAKAT_ARCFACE_HF`)

Point at any HF repo + file that hosts an IR-ResNet50 safetensors:

```bash
export PLAKAT_ARCFACE_HF=<user>/<repo>#<path/in/repo.safetensors>
```

plakat downloads + caches via the standard HF infrastructure. There's
no canonical default repo at the moment (no official InsightFace HF
safetensors mirror), so this route is a user-configured shortcut around
the ONNX-to-safetensors conversion. Discover candidates at
[huggingface.co/models?search=arcface+iresnet50](https://huggingface.co/models?search=arcface+iresnet50).

Either route works. Local file wins if both are set. Verify any time:

```bash
plakat doctor # offline check — parses + verifies local file existence
plakat doctor --verify # active — attempts the HF download to confirm
plakat doctor --json # v0.19 — structured report for CI / jq piping
```

`plakat doctor` also reports your build's backend features vs the
runtime-resolved device (v0.18) — useful when a CUDA binary
silently falls back to CPU because the NVIDIA driver isn't
installed (the driver shim `libcuda.so.1` is separate from the
CUDA toolkit, and a Linux + `--features cuda` doctor run probes
the well-known driver paths and flags a missing shim). The HF
cache disk usage is reported alongside, with cleanup hints
(`plakat models rm <repo>`) when it grows over 100 GB.

### Usage examples

```bash
# SD 1.5 FaceID with centre-crop alignment + auto-applied UNet LoRA.
plakat --device metal portrait \
 --photo ./refs/alice.jpg \
 --identity faceid \
 --face-strength 1.0 \
 --steps 30 \
 "studio portrait, cinematic lighting"

# User-supplied bbox — useful when the face is off-centre or the
# photo isn't a tight head-and-shoulders crop.
plakat --device metal portrait \
 --photo ./refs/alice.jpg \
 --identity faceid \
 --face-bbox 0.20,0.05,0.80,0.65 \
 --face-strength 1.0 \
 "studio portrait, cinematic lighting"

# SDXL FaceID — same ArcFace weights, different image-proj.
# UNet LoRA applied automatically.
plakat --device metal portrait \
 --photo ./refs/alice.jpg \
 --model sdxl --identity faceid-sdxl \
 --face-bbox 0.20,0.05,0.80,0.65 \
 --face-strength 1.0 \
 --size 832x1216 \
 "studio portrait, cinematic lighting"

# Disable the auto-applied FaceID UNet LoRA. Useful for A/B testing
# whether a particular prompt suffers under the shared-cross-attention
# application of the LoRA.
PLAKAT_FACEID_LORA=off plakat --device metal portrait \
 --photo ./refs/alice.jpg --identity faceid \
 --face-bbox 0.20,0.05,0.80,0.65 --face-strength 1.0 \
 "studio portrait"

# Best alignment available: 5-point ArcFace canonical similarity
# transform. Order: left_eye, right_eye, nose, left_mouth_corner,
# right_mouth_corner. Normalised to [0, 1] in the photo's space.
plakat --device metal portrait \
 --photo ./refs/alice.jpg \
 --identity faceid \
 --face-landmarks 0.40,0.40,0.60,0.40,0.50,0.55,0.42,0.68,0.58,0.68 \
 --face-strength 1.0 \
 "studio portrait, soft natural lighting"
```

---

## Optional SCRFD auto-detection

`PLAKAT_SCRFD_WEIGHTS` or `PLAKAT_SCRFD_HF` enables automatic face
detection — the SCRFD detector finds faces and fills in the 5-point
landmarks the aligner already consumes, so `--face-bbox` /
`--face-landmarks` flags become unnecessary for typical portrait photos.

### Route A — local file (`PLAKAT_SCRFD_WEIGHTS`)

```bash
# Download SCRFD-500MF from InsightFace releases
curl -L -o scrfd_500m_bnkps.onnx \
 https://github.com/deepinsight/insightface/releases/download//scrfd_500m_bnkps.onnx

# Convert ONNX → safetensors
python -c "import onnx, torch
from onnx2torch import convert
from safetensors.torch import save_file
m = convert(onnx.load('scrfd_500m_bnkps.onnx'))
save_file(m.state_dict(), 'scrfd_500m.safetensors')"

export PLAKAT_SCRFD_WEIGHTS=$(pwd)/scrfd_500m.safetensors
```

### Route B — HuggingFace-hosted (`PLAKAT_SCRFD_HF`)

```bash
export PLAKAT_SCRFD_HF=<user>/<repo>#<path/in/repo.safetensors>
```

Discover candidates at [huggingface.co/models?search=scrfd](https://huggingface.co/models?search=scrfd).
Local file wins if both are set.

> ⚠️ **Verification status (honest).** The SCRFD architecture in
> `src/pipelines/scrfd.rs` was ported from the InsightFace reference
> — exact channel widths / block counts for SCRFD-500MF are best-guess
> against the published architecture. If your converted safetensors
> uses a slightly different layout, weight loading will error at the
> first mismatched layer. `SCRFDConfig::scrfd_500mf()` is parametric,
> so fixes are usually a one-line change. File a bug with the actual
> layer that mismatched + its shape and we'll patch.

### Alignment priority

When multiple alignment sources are configured, the encoder picks the
richest available:

1. **5-point landmarks** (`--face-landmarks` / persona `face-landmarks`)
 — proper alignment via similarity transform. Captures ~95% of
 ArcFace's discriminative power.
2. **SCRFD auto-detected landmarks** (when `PLAKAT_SCRFD_*` is set and
 no manual landmarks were supplied) — same alignment quality as (1)
 without you having to type the coordinates.
3. **Bbox crop** (`--face-bbox` / persona `face-bbox`) — crop to the
 bbox, then resize to 112×112. No rotation/scale correction.
 ~75–85% of full power.
4. **Centre-crop fallback** — shorter-side resize + centre-crop. Works
 for tight head-and-shoulders photos; degrades on off-centre inputs.
 ~70–80% of full power.

### Where to get landmarks (when not using SCRFD)

`--face-landmarks` takes user-supplied points in the order
`left_eye, right_eye, nose, left_mouth_corner, right_mouth_corner`.

1. **InsightFace's `face_analysis.get()`** (Python) publishes 5
 landmarks per detected face in this exact order — copy them out.
2. **MediaPipe / dlib / similar** — subset to the 5 ArcFace points.
 For dlib's 68-point output: indices 36, 45, 30, 48, 54.
3. **Eyeball them.** For a head-and-shoulders portrait, rough guesses
 in normalised coords usually land within a few pixels of true:
 - eyes typically at `y ≈ 0.35–0.45`, separation `Δx ≈ 0.20`
 - nose at `y ≈ 0.50–0.60`, centred
 - mouth corners at `y ≈ 0.65–0.75`, separation `Δx ≈ 0.15`

 The similarity transform absorbs reasonable estimation noise — a
 few percent error in landmark position produces visually similar
 alignment.
