#import "lib.typ": *

#page-header(
  "Filesystem Layout",
  "Paths, environment variables, and configuration."
)

This page is the canonical reference for where `mx` reads and writes files,
which environment variables control those locations, and what to expect when
upgrading from an earlier release.

== Table of contents

- #link(<the-principle>)[The principle]
- #link(<layout>)[Layout]
- #link(<env-vars>)[Environment variables]
- #link(<migration>)[Migration & legacy fallbacks]
- #link(<renamed-commands>)[Renamed and removed CLI commands]
- #link(<examples>)[Examples]
- #link(<contributors>)[Notes for contributors]


// ═══════════════════════════════════════════════════════════════════════
// THE PRINCIPLE
// ═══════════════════════════════════════════════════════════════════════

== The principle <the-principle>

Every path `mx` touches derives from a single base directory: `$MX_HOME`, which
defaults to `~/.mx/`. Each subsystem owns a subdirectory beneath that base.

Overrides are layered. From most specific to least specific:

+ *Per-file* -- e.g. `MX_KV_SCHEMA`, `MX_KV_DATA`. These point at exact files
  (and may include a `{agent}` placeholder).
+ *Per-subsystem* -- e.g. `MX_SURREAL_ROOT`, `MX_CODEX_PATH`,
  `MX_ISOLATE_MODELS`. These move one subsystem's root.
+ *Base* -- `MX_HOME`. Moves the entire tree at once.
+ *Default* -- `~/.mx/`.

A more specific override always wins. Setting `MX_KV_DATA=/etc/foo.json` keeps
that one file at `/etc/foo.json` regardless of `MX_HOME` or any subsystem
override.

The base path is hardcoded in exactly one place:
#link("https://github.com/coryzibell/mx/blob/main/src/paths.rs")[`src/paths.rs`].
Everything else asks `paths.rs` for its location.


// ═══════════════════════════════════════════════════════════════════════
// LAYOUT
// ═══════════════════════════════════════════════════════════════════════

== Layout <layout>

```
~/.mx/                          # base; override $MX_HOME
├── kv/
│   ├── schema/{agent}.toml     # override MX_KV_SCHEMA
│   └── data/{agent}.json       # override MX_KV_DATA
├── state/
│   └── schemas/{id}.yaml       # default ID: "tensor"; CLI --schema flag
├── memory/
│   ├── surreal/                # override MX_SURREAL_ROOT
│   ├── embed/                  # only when MX_ISOLATE_MODELS is set
│   └── seed/
│       ├── agents/             # *.md (frontmatter)
│       └── knowledge/          # *.jsonl (markdown ingest tracked in #257)
├── codex/                      # override MX_CODEX_PATH
├── cache/sync/{owner-repo}/
├── artifacts/
└── swap/
```

=== `kv/` <layout-kv>

The KV store: per-agent schema (TOML) and data (JSON). Used by `mx kv` and any
agent that needs fast local state. Each agent gets one schema file and one data
file, keyed off the #link(<env-other>)[`MX_CURRENT_AGENT`] environment variable.

- `kv/schema/{agent}.toml` -- TOML schema declaring keys, types, and defaults.
- `kv/data/{agent}.json` -- JSON-serialized current values, written atomically.

Override either file with #link(<env-paths>)[`MX_KV_SCHEMA`] /
#link(<env-paths>)[`MX_KV_DATA`]. Both env vars accept the literal string
`{agent}`, which is substituted with the active agent name at resolution time.

=== `state/schemas/` <layout-state>

YAML (or JSON) schemas for the emotional-state tensor system used by
`mx state`. The default schema ID is `tensor`, resolving to
`state/schemas/tensor.yaml`.

Pick a different schema with `mx state ... --schema {id|path}`. The flag
argument is classified as a path or an ID by a simple heuristic, then looked
up:

- *Bare ID* (e.g. `--schema tensor`): the loader tries
  `state/schemas/{id}.yaml` first, then `state/schemas/{id}.yml`, then
  `state/schemas/{id}.json`. The first file that exists wins; if none exist,
  the lookup fails with a "schema not found" error.
- *Direct path* (e.g. `--schema /tmp/foo.json`): the file is loaded
  directly. Any extension is accepted; the parser tries YAML first and falls
  back to JSON based on file contents, not the extension.

The argument is classified as a path if it contains a `/` *or* ends with
`.yaml`, `.yml`, or `.json`; otherwise it is classified as a bare ID. A dot
elsewhere in the name is irrelevant -- only those three suffixes flip the
classification. So `--schema my.schema` is treated as an *ID* (no slash,
no recognized extension) and, if no matching file exists under
`state/schemas/`, fails with a "schema not found" error. To force a path
lookup of a dotted name without one of the recognized extensions, prefix it
with `./` (e.g. `--schema ./my.schema`) -- the slash flips it to path mode.

There is no env-var override for the schema choice anymore -- the old
`MX_STATE_SCHEMA` was replaced by the CLI flag. (See
#link(<renamed-commands>)[Renamed and removed CLI commands].)

=== `memory/` <layout-memory>

The knowledge graph backend and its inputs.

- `memory/surreal/` -- SurrealKV embedded database files. Override the whole
  directory with #link(<env-paths>)[`MX_SURREAL_ROOT`]. For network-mode
  SurrealDB, the directory is unused -- see the
  #link(<env-surreal>)[SurrealDB connection vars].
- `memory/embed/` -- only created when #link(<env-paths>)[`MX_ISOLATE_MODELS`]
  (or legacy `MX_ISOLATE_FASTEMBED`) is set. Holds downloaded ONNX model and
  tokenizer files that would otherwise live in the shared XDG cache
  (`$XDG_CACHE_HOME/huggingface/`). Use this when you want mx's model cache
  isolated from other tools.
- `memory/seed/agents/` -- markdown files with YAML frontmatter, one per agent.
  Loaded by `mx memory seed agents`.
- `memory/seed/knowledge/` -- one or more `*.jsonl` files. Loaded by
  `mx memory seed knowledge`, which scans the directory for every `.jsonl` it
  finds.

=== `codex/` <layout-codex>

Session archives written by `mx codex archive` -- transcripts, extracted images,
and per-archive manifests. Override with #link(<env-paths>)[`MX_CODEX_PATH`].
This is typically the largest directory in `~/.mx/`; point it at a roomier disk
if you archive a lot of sessions.

=== `cache/sync/{owner-repo}/` <layout-cache>

Per-repo cache directory used by `mx sync` to track GitHub issues and
discussions across runs. The repo slug replaces the `/` in `owner/repo` with
`-` (so `coryzibell/mx` becomes `coryzibell-mx`). Safe to delete -- it will be
rebuilt on the next `mx sync pull`.

=== `artifacts/` <layout-artifacts>

Generic output directory for handlers that need to drop a file somewhere
predictable but don't have a more specific home. Treat as ephemeral.

=== `swap/` <layout-swap>

Scratch space for in-flight operations. Cleared opportunistically; do not store
anything you want to keep.


// ═══════════════════════════════════════════════════════════════════════
// ENVIRONMENT VARIABLES
// ═══════════════════════════════════════════════════════════════════════

== Environment variables <env-vars>

=== Path overrides <env-paths>

#table(
  columns: (auto, auto, auto, 1fr),
  table.header([*Variable*], [*Type*], [*Default*], [*Overrides*]),
  [`MX_HOME`], [path], [`~/.mx/`], [The base directory; moves the entire tree],
  [`MX_SURREAL_ROOT`], [path], [`$MX_HOME/memory/surreal/`], [SurrealKV embedded-database root],
  [`MX_CODEX_PATH`], [path], [`$MX_HOME/codex/`], [Codex archive directory],
  [`MX_KV_SCHEMA`], [path template], [`$MX_HOME/kv/schema/{agent}.toml`], [KV schema file; `{agent}` placeholder substituted],
  [`MX_KV_DATA`], [path template], [`$MX_HOME/kv/data/{agent}.json`], [KV data file; `{agent}` placeholder substituted],
  [`MX_ISOLATE_MODELS`], [boolean flag], [unset], [When non-empty, redirects the model cache from XDG to `$MX_HOME/memory/embed/`. Legacy `MX_ISOLATE_FASTEMBED` is also honored],
)

Empty-string values for the path overrides in this table (`MX_HOME`,
`MX_SURREAL_ROOT`, `MX_CODEX_PATH`, `MX_KV_SCHEMA`, `MX_KV_DATA`,
`MX_ISOLATE_MODELS`) are treated as unset and fall back to the default.

The same is *not* uniformly true of other `MX_*` env vars. In particular,
the #link(<env-surreal>)[SurrealDB connection vars] below do not all filter
empty strings: setting `MX_SURREAL_USER=""` produces an empty username, not the
default `root`. Among the connection vars only `MX_SURREAL_PASS` /
`MX_SURREAL_PASS_FILE` are empty-filtered. To restore a default, *leave the
variable unset entirely* rather than setting it to an empty string.

The boolean flag (`MX_ISOLATE_MODELS`) is "on" for any non-empty value
(`1`, `true`, `yes` -- it doesn't parse, it just checks for non-emptiness).

=== SurrealDB connection <env-surreal>

These configure the SurrealDB driver. They affect which database mx talks to,
not where files live on disk (with the exception of `MX_SURREAL_ROOT` above,
which is the embedded-mode storage location).

#table(
  columns: (auto, auto, auto, 1fr),
  table.header([*Variable*], [*Type*], [*Default*], [*Purpose*]),
  [`MX_SURREAL_MODE`], [enum], [`embedded`], [`embedded` for local SurrealKV, `network` for WebSocket],
  [`MX_SURREAL_URL`], [URL], [`ws://localhost:8000`], [WebSocket URL (network mode only)],
  [`MX_SURREAL_USER`], [string], [`root`], [Username],
  [`MX_SURREAL_PASS`], [secret], [unset], [Password (literal value)],
  [`MX_SURREAL_PASS_FILE`], [path], [unset], [Path to a file containing the password (e.g. an agenix secret); read when `MX_SURREAL_PASS` is unset],
  [`MX_SURREAL_NS`], [string], [`memory`], [Namespace],
  [`MX_SURREAL_DB`], [string], [`knowledge`], [Database name],
  [`MX_SURREAL_AUTH_LEVEL`], [enum], [`root`], [One of `root`, `namespace` (or `ns`), `database` (or `db`)],
  [`MX_SKIP_SCHEMA`], [boolean], [unset], [Set to `1` or `true` to skip automatic schema application on connection. Escape hatch for restricted DB permissions. Ignored by `mx migrate`, which always applies the schema],
)

=== GitHub App auth (sync) <env-github>

Optional. Only needed when `mx sync` runs against a private repo via a GitHub
App rather than via your personal `gh` token.

#table(
  columns: (auto, auto, auto, 1fr),
  table.header([*Variable*], [*Type*], [*Default*], [*Purpose*]),
  [`MX_GITHUB_APP_ID`], [string], [unset], [GitHub App ID],
  [`MX_GITHUB_INSTALLATION_ID`], [string], [unset], [App installation ID for the target org/user],
  [`MX_GITHUB_PRIVATE_KEY`], [secret (PEM)], [unset], [App private key, PEM-encoded],
)

=== Identity & display <env-other>

#table(
  columns: (auto, auto, auto, 1fr),
  table.header([*Variable*], [*Type*], [*Default*], [*Purpose*]),
  [`MX_CURRENT_AGENT`], [string], [unset], [Active agent identity. Required for `mx memory wake` and any command that reads/writes per-agent KV. Also the default for `--source-agent` on `mx memory add`],
  [`MX_USER_NAME`], [string], [`git config user.name`, else `"User"`], [Display name for "user" turns in codex transcripts. Resolution order: env var > git config > literal `"User"`],
  [`MX_ASSISTANT_NAME`], [string], [`"Orchestrator"`], [Display name for "assistant" turns in codex transcripts. No git fallback -- the default is the literal string `Orchestrator`],
)

=== Tuning <env-tuning>

#table(
  columns: (auto, auto, auto, 1fr),
  table.header([*Variable*], [*Type*], [*Default*], [*Purpose*]),
  [`MX_WAKE_CHUNK_BYTES`], [integer], [`28000`], [Maximum bytes per chunk during the wake-ritual presentation step. Values that fail to parse or are zero fall back silently to the default],
)

=== Removed <env-removed>

#table(
  columns: (auto, 1fr),
  table.header([*Variable*], [*Replacement*]),
  [`MX_MEMORY_PATH`], [Use #link(<env-paths>)[`MX_SURREAL_ROOT`] for just the database, or #link(<env-paths>)[`MX_HOME`] to move everything together. Setting the old name now emits a one-line stderr note and is otherwise ignored],
  [`MX_STATE_SCHEMA`], [Use the `mx state ... --schema {id|path}` CLI flag. The default schema ID also changed: it is now `tensor` (was `crewu`)],
)


// ═══════════════════════════════════════════════════════════════════════
// MIGRATION & LEGACY FALLBACKS
// ═══════════════════════════════════════════════════════════════════════

== Migration & legacy fallbacks <migration>

The path-alignment refactor (\#255, merged via PR \#259) moved several files
without breaking older installs. For one release cycle, mx will read from the
old locations as a soft fallback and emit a one-line `note:` to stderr telling
you what moved. *No data is lost. The warnings are informative, not errors.*

#table(
  columns: (1fr, 1fr, 1fr),
  table.header([*Legacy location*], [*New location*], [*Behavior*]),
  [`~/.crewu/kv/{agent}.schema.toml`, `~/.crewu/kv/{agent}.data.json`],
    [`$MX_HOME/kv/schema/{agent}.toml`, `$MX_HOME/kv/data/{agent}.json`],
    [Read-only fallback; consolidated stderr note fires once per process],
  [`$MX_HOME/agents/` (agent seed `*.md`)],
    [`$MX_HOME/memory/seed/agents/`],
    [Read-only fallback; stderr note when used],
  [`$MX_HOME/memory/index.jsonl` (knowledge seed)],
    [`$MX_HOME/memory/seed/knowledge/*.jsonl`],
    [Read-only fallback. This is a _shape_ change, not a rename: the old location was a single hardcoded file (`index.jsonl`); the new location is a directory scanned for every `*.jsonl` it finds. Stderr note when the legacy file is read],
  [`MX_MEMORY_PATH` env var],
    [`MX_SURREAL_ROOT` env var],
    [Old var *not honored*; setting it just triggers a rename note],
)

To silence the warnings, move the files (or rename the env var). The fallbacks
will be removed in a future release. To track the removal in source, grep for
`TODO(*-migration)` and `TODO(memory-path-rename-note)` in the codebase.


// ═══════════════════════════════════════════════════════════════════════
// RENAMED AND REMOVED CLI COMMANDS
// ═══════════════════════════════════════════════════════════════════════

== Renamed and removed CLI commands <renamed-commands>

#table(
  columns: (auto, auto, 1fr),
  table.header([*Old*], [*New*], [*Notes*]),
  [`mx agents seed`], [`mx memory seed agents`], [Old form still parses but bails with a one-line pointer to the new command],
  [`mx memory import`], [`mx memory seed knowledge`], [Now scans a directory; loads every `*.jsonl` it finds rather than a single hardcoded file],
  [`mx memory rebuild`], [(removed)], [Reindexing moved out of the user-facing surface; see issue \#258 for `mx doctor memory rebuild`],
  [`mx state ... --env-MX_STATE_SCHEMA`], [`mx state ... --schema {id|path}`], [CLI flag replaces the env var; accepts a bare schema ID or a direct path],
  [`codex save`], [`codex archive`], [Renamed for clarity],
  [`session export`], [`codex export`], [Moved under the codex subcommand],
)

#deprecated[The old command names in this table still parse in current builds
but emit a pointer to their replacement. They will be removed in a future
release.]


// ═══════════════════════════════════════════════════════════════════════
// EXAMPLES
// ═══════════════════════════════════════════════════════════════════════

== Examples <examples>

Move the entire mx tree to a different disk:

```bash
export MX_HOME=/data/mx
```

Keep mx's defaults but put the SurrealDB store on a fast SSD:

```bash
export MX_SURREAL_ROOT=/mnt/ssd/mx-surreal
```

Use a custom KV schema for the `inkwell` agent (without overriding the data
file):

```bash
export MX_KV_SCHEMA=/etc/mx/inkwell-schema.toml
```

Use a path template that resolves per-agent (one variable, many agents):

```bash
export MX_KV_SCHEMA='/etc/mx/schemas/{agent}.toml'
export MX_KV_DATA='/var/lib/mx/{agent}.json'
```

Isolate the model cache so it doesn't share with other tools:

```bash
export MX_ISOLATE_MODELS=1
# Models will now download into $MX_HOME/memory/embed/
```

Encode a tensor. The first form uses the default `tensor` schema; the second
points at an explicit schema file:

```bash
mx state encode --dimensions "temp=0.8 entropy=0.75 agency=0.4"
mx state encode --schema /tmp/myschema.yaml -d "temp=0.5"
```

To target a non-default schema by ID, drop a YAML file at
`$MX_HOME/state/schemas/{id}.yaml` and pass `--schema {id}`. The bare-ID
form is what the lookup helper handles; for an absolute or relative file
path, just pass the path directly (see
#link(<layout-state>)[`state/schemas/`] for the path-vs-ID heuristic).

Point SurrealDB at a remote network instance (schema is applied automatically
on connection, just like embedded mode):

```bash
export MX_SURREAL_MODE=network
export MX_SURREAL_URL=ws://surreal.internal:8000
export MX_SURREAL_USER=mx
export MX_SURREAL_PASS_FILE=/run/agenix/mx-surreal-pass
```

Skip auto-apply when the DB user lacks DDL permissions (schema managed
externally by an admin):

```bash
export MX_SKIP_SCHEMA=1
# To explicitly apply schema when needed, use:
# mx migrate
```


// ═══════════════════════════════════════════════════════════════════════
// NOTES FOR CONTRIBUTORS
// ═══════════════════════════════════════════════════════════════════════

== Notes for contributors <contributors>

Every path in mx routes through
#link("https://github.com/coryzibell/mx/blob/main/src/paths.rs")[`src/paths.rs`].
New helpers follow the `_with(env_val: Option<&str>, home: &Path)` test-seam
pattern -- see `paths::codex_dir_with` for the canonical example. The pattern
keeps resolution logic pure: tests call the `_with` variant directly with
explicit arguments and never mutate process env state, so the suite runs safely
in parallel.

Two rules:

+ Do not call `dirs::home_dir()` outside `src/paths.rs`. If you need a
  home-relative path, add a helper to `paths.rs` and call it from your module.
  `paths.rs` itself is the _only_ legitimate caller of `dirs::home_dir()` in
  the tree. It uses it for: `mx_home()` (the `~/.mx/` default),
  `legacy_crewu_kv_schema_path` and `legacy_crewu_kv_data_path` (the legacy
  fallbacks for the kv migration), and `claude_projects_dir` /
  `claude_config_path` (read-only locations owned by another tool, Claude).
  Anything new that needs `home_dir()` -- including helpers for paths owned
  by other tools -- belongs in `paths.rs` too, so this rule stays absolute
  everywhere else. Do not "fix" the existing calls for consistency; they are
  the carve-out.
+ Do not read `MX_*` env vars in handlers if a path helper already encapsulates
  that override. Add the env-var read inside the helper instead, behind the
  `_with` seam.
