<p align="center"><a href="https://gitlab.com/flo-labs/pub/dms"><img src="https://gitlab.com/flo-labs/pub/dms-rs/-/raw/main/assets/logo.png" alt="DMS" width="120"></a></p>

# dms-rs

Rust reference parser for **[DMS](https://gitlab.com/flo-labs/pub/dms)**, a data syntax with strong typing, ordered maps,
multi-line heredocs, and front-matter metadata.

This crate is the canonical implementation. Six other ports (C, Zig, Go, Python,
JavaScript, Perl) check against the same fixture corpus.

## What DMS looks like

A medium-size tier-0 document, exercising every feature you'd touch in a
real config — front matter, comments (line + trailing), nested tables,
list-of-tables with the `+` marker, flow forms, distinct types, and a
heredoc with a trim modifier:

```dms
+++
title:    "DMS feature tour"
version:  "1.0.0"
updated:  2026-04-24T09:30:00-04:00
+++

# Hash and // line comments both work.
// Bare keys allow full Unicode; quoted keys take any string.

database:
  host:    "db.internal"
  port:    5432            # bumped after the LB change
  pool:    { size: 10, idle_timeout_s: 30 }   # flow table

servers:
  + name: "web1"
    disks:
      + mount: "/"
        size_gb: 100
      + mount: "/var"
        size_gb: 500
  + name: "web2"

regions: ["us-east-1", "eu-west-1", "ap-south-1"]

sql: """SQL _trim("\n", ">")
    SELECT id, email
      FROM users
     WHERE active = true
    SQL
```

Tier 1 layers structured decorators on top of the value tree. Sigils bind
to families published by a dialect; here is `dms+html` carrying an HTML
fragment as a DMS document:

```dms
+++
_dms_tier: 1
_dms_imports:
  + dialect: "html"
    version: "1.0.0"
+++

+ |html(lang: "en")
  + |head
    + |title "DMS feature tour"
    + |meta(charset: "UTF-8")
  + |body(class: "main")
    + |h1 "Welcome to DMS"
    + |p(class: "lede")
      + "Click "
      + |a(href: "/spec.html") "here"
      + " to read the spec."
```

Full feature tour, format comparison, and dialect index on the
**[DMS website](https://flo-labs.gitlab.io/pub/dms-webpage/)**.

## Install

```toml
# Cargo.toml — note the rename so source code can still `use dms::...`
[dependencies]
dms = { package = "dms-rs", version = "0.4" }
```

Or:

```sh
cargo add dms-rs --rename dms
```

## Usage

```rust
use dms::{decode, decode_document, encode};

let src = std::fs::read_to_string("config.dms")?;

// Body-only (drops front matter and comments after decode).
let body = decode(&src)?;

// Full document (preserves comments + literal forms for encode round-trip).
let doc = decode_document(&src)?;
let _meta = doc.meta.as_ref();           // Option<&IndexMap<String, Value>>
let _root = &doc.body;                   // dms::Value
// doc.comments and doc.original_forms carry the round-trip metadata.

// Re-emit DMS source. `encode` returns `Result<String, EncodeError>`
// — full-mode emit refuses a `Document` containing an unordered table
// (use `encode_lite` for that case).
let out = encode(&doc)?;
```

### Front-matter-only decode

For callers that need only the document's metadata — config loaders
checking `_dms_tier`, indexers harvesting user keys, dispatchers
choosing a downstream decoder — `decode_front_matter` parses the
`+++ ... +++` block and stops, leaving body bytes untokenized. SPEC
tier 0 requires this entry point. Validation inside the FM block is
identical to a full decode (open/close on their own lines, `_`-prefix
namespace enforced, unterminated FM is an error); body errors are
silently skipped.

```rust
use dms::{decode_front_matter, Value};

match decode_front_matter(&src)? {
    None => { /* document has no `+++` front-matter block */ }
    Some(Value::Table(meta)) => {
        // empty Table => present-but-empty FM (`+++\n+++`),
        // distinguishable from `None` above.
        if let Some(Value::String(title)) = meta.get("title") {
            println!("title: {title}");
        }
    }
    Some(_) => unreachable!("FM is always a table"),
}
```

> **Migration notes (v0.4):**
>
> - Tier 1 lands in this release. New module `dms::tier1` provides
>   `decode_t1`, `encode_t1`, `decode_t1_with_registry`,
>   `encode_t1_with_registry`, plus dialect registry types.
>   `dms::tier1_stream` adds the streaming API. Tier-0 surface
>   unchanged.
> - Validation error messages use friendly type names
>   (`string` / `integer` / ...) instead of internal discriminants.
> - `dms_html_dialect()` helper returns a ready-to-use html dialect
>   spec for callers that don't want to construct one by hand.
> - `mutate_t1` operations on `DocumentT1` (`insert_at`, `remove_at`,
>   `replace_at`) keep the decorators sidecar aligned with structural
>   body changes.

> **Migration notes (v0.3):**
>
> - SPEC v0.14 renamed `parse` / `to_dms` to `decode` / `encode`. The
>   old names still work as deprecated thin aliases for one release;
>   new code should use `decode` / `encode`.
> - `ParseError` has been renamed to `DecodeError`; `pub type
>   ParseError = DecodeError` keeps the old name working as a
>   deprecated alias.
> - `encode` / `encode_with_mode` now return `Result<String,
>   EncodeError>` instead of `String`. Previously full-mode emit
>   panicked on a Document containing `Value::UnorderedTable`; the
>   panic is now a clean `Err(EncodeError::UnorderedInFullMode)`.
>   `encode_lite` still returns `String` (lite emit accepts unordered
>   input and cannot fail).

## Value shape

| DMS type        | Rust type                              |
| --------------- | -------------------------------------- |
| bool            | `Value::Bool(bool)`                    |
| integer         | `Value::Integer(i64)`                  |
| float           | `Value::Float(f64)`                    |
| string          | `Value::String(String)`                |
| local-date      | `Value::LocalDate(String)`             |
| local-time      | `Value::LocalTime(String)`             |
| local-datetime  | `Value::LocalDateTime(String)`         |
| offset-datetime | `Value::OffsetDateTime(String)`        |
| table           | `Value::Table(IndexMap<String, Value>)`|
| list            | `Value::List(Vec<Value>)`              |

Datetime variants carry the source lexeme as a `String` — already
SPEC-validated, so you never re-parse to inspect them. Tables use
`indexmap::IndexMap` to preserve insertion order. The `IndexMap` type
is re-exported from this crate so downstream code can build meta
tables without its own dependency on `indexmap`.

## Working with comments and heredocs

DMS preserves comments through decode → mutate → re-emit (SPEC
§Comments). The `Document` carries them on a side-channel keyed by
breadcrumb path; the same shape lets you attach a comment to a value
*after* decoding and have it round-trip through `encode`:

```rust
use dms::{decode_document, encode,
          AttachedComment, Comment, CommentKind, CommentPosition,
          BreadcrumbSegment, Value};

let mut doc = decode_document("db:\n  port: 8080\n")?;

// Mutate a value in place.
if let Value::Table(ref mut t) = doc.body {
    if let Some(Value::Table(ref mut db)) = t.get_mut("db") {
        db.insert("port".into(), Value::Integer(5432));
    }
}

// Attach a leading line comment to db.port.
doc.comments.push(AttachedComment {
    comment: Comment {
        content: "# bumped after LB change".into(),
        kind: CommentKind::Line,
    },
    position: CommentPosition::Leading,
    path: vec![
        BreadcrumbSegment::Key("db".into()),
        BreadcrumbSegment::Key("port".into()),
    ],
});

println!("{}", encode(&doc));
```

### Forcing a heredoc on emit

Strings parse and re-emit in their source form. To switch a basic-quoted
string to a heredoc (or to construct one from scratch), push an
`OriginalLiteral::String` record onto `doc.original_forms` keyed by the
value's path:

```rust
use dms::{OriginalLiteral, StringForm, HeredocFlavor, BreadcrumbSegment, Value};

if let Value::Table(ref mut t) = doc.body {
    if let Some(Value::Table(ref mut db)) = t.get_mut("db") {
        db.insert("greeting".into(),
                  Value::String("Hello, friend.\nWelcome aboard.\n".into()));
    }
}

doc.original_forms.push((
    vec![
        BreadcrumbSegment::Key("db".into()),
        BreadcrumbSegment::Key("greeting".into()),
    ],
    OriginalLiteral::String {
        form: StringForm::Heredoc {
            flavor: HeredocFlavor::BasicTriple,   // or LiteralTriple for '''
            label: None,                          // or Some("EOF".into())
            modifiers: vec![],                    // _trim(...), _fold_paragraphs(), …
        },
    },
));
```

Round-trip rules (SPEC §Round-trip semantics): comments stick to
*still-present* nodes; deleting a node drops its comments; newly
inserted nodes start with no comments. The first
`original_forms` entry per path wins, so override the parser-recorded
form by replacing rather than appending if the key is already present.

## Tier 1: decorators and dialects

Tier-1 source carries dialect imports + decorator calls (`|tag(...)`,
`@expr(...)`, etc.). See [TIER1.md](https://gitlab.com/flo-labs/pub/dms/-/blob/main/TIER1.md)
for the full spec. dms-rs ships a feature-complete batch decoder /
encoder.

```rust
use dms::tier1::{
    decode_t1, decode_t1_with_registry,
    encode_t1, encode_t1_with_registry,
    DialectSpec, FamilySpec, InMemoryRegistry, ParamSignature,
    VersionStrategy,
};

let src = std::fs::read_to_string("page.dms.html")?;

// Parse without a registry — decorators are captured into the sidecar
// but content_slot hoisting and family-signature validation are skipped.
let doc = decode_t1(&src)?;
let _decorations = &doc.decorators;       // Vec<DecoratorEntry>
let _imports     = &doc.imports;          // ResolvedImports

// With a dialect registry — full hoist + alias / allow / deny resolution
// + param-signature validation including positional / variadic / named
// structs.
let registry = InMemoryRegistry::new().with(DialectSpec {
    name: "html".into(),
    version: "1.0.0".into(),
    version_strategy: VersionStrategy::default(),  // caret
    families: vec![FamilySpec {
        name: "tag".into(),
        default_sigils: vec!["|".into()],
        empty_default: dms::Value::Table(Default::default()),
        content_slot: Some("children".into()),
        params: ParamSignature::default(),
    }],
    structs: vec![],
});
let doc = decode_t1_with_registry(&src, &registry)?;

// Encoder canonical form: short inline spans re-fold into the
// content_slot param; empty-default body matches suppress redundant
// `: {}` emission.
let out = encode_t1_with_registry(&doc, &registry)?;
```

**Dialect versioning:** five strategies (`exact` / `caret` / `tilde`
/ `gte` / `any`), default `caret`. Pre-release tags participate in
matching per semver; build metadata ignored. Range-specifier syntax
(`^1.0.0` / `~1.0.0`) in a file's `version:` field is rejected — write
a plain semver string, the dialect declares its own match strategy.

### Streaming decode

`tier1_stream::decode_t1_streaming` exposes a `StreamEvent` iterator
over the parsed document — useful for SAX-style consumers that want
per-node callbacks instead of walking the value tree manually.

```rust
use dms::tier1_stream::{decode_t1_streaming, StreamEvent, ContainerKind};

for event in decode_t1_streaming(&src)? {
    match event {
        StreamEvent::FrontMatter { meta }            => { /* once at start */ }
        StreamEvent::EnterContainer { path, kind, decorations } => {
            // kind = Table | List
        }
        StreamEvent::Leaf { path, value, decorations } => { /* scalar node */ }
        StreamEvent::LeaveContainer { path, kind }   => { /* DEDENT */ }
        StreamEvent::EndOfDocument { root_decorations } => { /* once at end */ }
    }
}
```

Event order follows TIER1.md §"Streaming / incremental decode" yield
points: containers fire `EnterContainer` before children and
`LeaveContainer` after; leaves fire `Leaf` once with their decoration
sidecar attached; flow forms (`[...]` / `{...}`) materialize into the
parent container's value rather than firing inner events.

**Caveat — not a coroutine streamer.** dms-rs is batch-only per
TIER1.md §"Streaming / incremental decode" (the spec explicitly allows
this). The streaming module post-processes the fully-decoded
`DocumentT1` into the event sequence — the parsing cost is paid up
front, then events drain at memcpy speed. This gives consumers the
streaming API contract without rewriting the parser; trade-off is
peak memory (whole doc held during iteration) versus a true
incremental decoder.

## Crates in this workspace

| crate         | crates.io      | purpose                                          |
| ------------- | -------------- | ------------------------------------------------ |
| `crates/dms`         | **`dms-rs`**   | the parser                                       |
| `crates/dms-encoder` | `dms-encoder`  | DMS → tagged-JSON encoder (conformance suite)    |
| `crates/serde_dms`   | `serde_dms`    | Serde `Deserializer` adapter                     |

## Build & test

```sh
cargo build --workspace
cargo test  --workspace
```

## Conformance

The fixture corpus lives in [dms-tests](https://gitlab.com/flo-labs/pub/dms-tests)
(4500+ pairs). Clone it once as a sibling:

```sh
git clone https://gitlab.com/flo-labs/pub/dms-tests.git ../dms-tests
```

Then build the encoder and run the sweep:

```sh
cargo build --release -p dms-encoder
python3 ../dms-tests/run_conformance.py target/release/dms-encoder
```

`dms-tests` can also drive every implementation in one shot — see its
README for the cross-language workflow.

## Publish

```sh
cargo publish -p dms-rs           # parser
cargo publish -p dms-encoder      # depends on dms-rs being published
cargo publish -p serde_dms        # ditto
```

You'll need `cargo login <token>` once. Publishing must be in dependency order
(`dms-rs` → `dms-encoder` / `serde_dms`).

## License

MIT OR Apache-2.0
