# `docspec-http`

Library providing an Axum-based HTTP server that wraps a DocSpec conversion pipeline. Designed for embedding in custom binaries or running via the unified `docspec http` subcommand.

Send markdown (`Content-Type: text/markdown`), HTML (`Content-Type: text/html`), or DOCX (`Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document`), receive BlockNote JSON (default), HTML (`Accept: text/html`), oxa.dev JSON (`Accept: application/vnd.oxa+json`), or Pandoc native (`Accept: application/vnd.pandoc.native`). The underlying DocSpec pipeline is streaming, but this v1 HTTP wrapper **buffers the request body and the conversion output in memory** before responding. End-to-end streaming over HTTP is planned for a future version. For now, request size scales with available memory.

> **HTML is paragraph-only.** The HTML reader currently parses `<p>` elements only, and the HTML writer currently emits only paragraph events. Other HTML input elements and non-paragraph output events (headings, lists, tables, formatting, etc.) are silently dropped. See [docspec-html-reader](../docspec-html-reader/README.md) and [docspec-html-writer](../docspec-html-writer/README.md).

## When to Use This Crate

Use `docspec-http` directly when you're embedding the HTTP server into your own binary or need programmatic control over server startup, configuration, or telemetry initialization.

For end users who want a running HTTP server without writing Rust code, install `docspec-cli` and run `docspec http`.

## Quick Start

```toml
[dependencies]
docspec-http = "1.5"
tokio = { version = "1", features = ["full"] }
```

```rust
use docspec_http::{serve, server::ServerConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ServerConfig { host: "127.0.0.1".to_string(), port: 3000 };
    serve(config).await?;
    Ok(())
}
```

Once the server is running, send documents with curl:

```bash
# Convert Markdown to BlockNote JSON
curl -X POST \
     -H 'Content-Type: text/markdown' \
     -d '# Hello' \
     http://localhost:3000/conversion

# Convert HTML to BlockNote JSON
curl -X POST \
     -H 'Content-Type: text/html' \
     -d '<p>Hello</p>' \
     http://localhost:3000/conversion

# Convert DOCX to BlockNote JSON
curl -X POST \
     -H 'Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document' \
     --data-binary @document.docx \
     http://localhost:3000/conversion
```

## Public API

| Item | Kind | Description |
| ---- | ---- | ----------- |
| `serve(config: ServerConfig)` | async fn | Start the HTTP server with the given configuration. Runs until SIGINT or SIGTERM. |
| `ServerConfig` | struct | Server bind configuration: `host: String`, `port: u16`. |
| `cli::Args` | struct | Clap-derived argument struct for `--host` and `--port`. Useful when building a CLI that delegates to this library. |
| `init_telemetry(dsn: Option<&str>)` | fn | Initialize Sentry integration. Call before `serve()` if you want error reporting. |
| `init_tracing()` | fn | Initialize the tracing subscriber (stderr, INFO, pretty format). Call before `serve()`. |
| `TelemetryGuard` | struct | Drop guard returned by `init_telemetry()`. Keep alive for the duration of the process to ensure Sentry flushes on exit. |

## Endpoints

| Method  | Path        | Description                                                      |
| ------- | ----------- | ---------------------------------------------------------------- |
| POST    | /conversion | Convert markdown, HTML, or DOCX to BlockNote (default), HTML, oxa.dev JSON, or Pandoc native |
| OPTIONS | /conversion | Preflight / allowed methods                                      |
| GET     | /health     | Liveness check                                                   |
| HEAD    | /health     | Liveness check (no body)                                         |
| OPTIONS | /health     | Allowed methods                                                  |
| GET     | /metrics    | Prometheus metrics in exposition format 0.0.4                    |

## Request / Response Headers

**`X-Request-ID`**: Generated (UUID v4) if the request omits it. Echoed back unchanged if present.

**`X-Trace-ID`**: Echoed back if present. Never generated by the server.

**`Cache-Control`**: `max-age=0, private, must-revalidate` on every response, including errors.

## Error Responses

All errors use RFC 7807 Problem Details JSON (`application/problem+json; charset=utf-8`).

| Code | Meaning                                                  |
| ---- | -------------------------------------------------------- |
| 400  | Empty body or invalid UTF-8                              |
| 404  | Unknown path                                             |
| 405  | Wrong method (response includes `Allow` header)          |
| 406  | `Accept` header excludes all supported output types      |
| 415  | `Content-Type` must be `text/markdown`, `text/html`, or `application/vnd.openxmlformats-officedocument.wordprocessingml.document` |
| 422  | Input parse error (malformed markdown, HTML, or DOCX)    |
| 500  | Internal conversion error                                |

Accepted `Accept` values for `/conversion`: `text/html` (HTML), `application/vnd.oxa+json` (oxa.dev), `application/vnd.pandoc.native` (Pandoc native), `application/vnd.docspec.blocknote+json`, `application/vnd.blocknote+json` (BlockNote alias), `application/*`, or `*/*`. Wildcards and missing `Accept` default to BlockNote for back-compat. Anything else returns 406.

## Deployment Notes

**TLS**: Use a reverse proxy (nginx, Caddy). The server speaks plain HTTP.

**CORS**: Use a reverse proxy. No CORS headers are added.

**Auth**: Use a reverse proxy or upstream gateway.

**Body size**: No limit. Large documents are accepted. DoS risk is accepted. Both the request body and the conversion output are held in memory for the duration of the request.

**Request timeout**: No timeout. Slow clients can hang a connection indefinitely.

## Logging

Logs go to stderr at INFO level in pretty format. There are no flags to change the log level or format.

## Observability

`docspec-http` integrates with [Sentry](https://sentry.io/) for error reporting.
Activation is fully opt-in via environment variables — the binary has zero Sentry
overhead when no DSN is configured.

### Activation

Set ONE of the following to enable Sentry:

- `DOCSPEC_SENTRY_DSN` — docspec-specific override (preferred)
- `SENTRY_DSN` — Sentry's standard convention (fallback)

If both are set, `DOCSPEC_SENTRY_DSN` wins. An empty string or malformed DSN is
treated as "not set" — the server starts normally and logs a warning to stderr.

### Configuration (all optional)

These follow Sentry's standard conventions:

- `SENTRY_ENVIRONMENT` — environment name (default: `production`)
- `SENTRY_RELEASE` — release identifier (default: auto, `docspec-http@<version>`)
- `SENTRY_SAMPLE_RATE` — error sample rate `[0.0, 1.0]` (default: `1.0`)
- `SENTRY_TRACES_SAMPLE_RATE` — performance trace sample rate `[0.0, 1.0]` (default: `0.0`, traces disabled)

### What is captured

| Signal                                                  | Sentry?                                 | PostHog?                  |
| ------------------------------------------------------- | --------------------------------------- | ------------------------- |
| `500 Internal Server Error` (`HttpError::Internal`)     | yes (event)                             | yes (`$exception`)        |
| `422 Unprocessable Entity` (`HttpError::Unprocessable`) | yes (event)                             | yes (`$exception`)        |
| Other 4xx responses                                     | no                                      | no                        |
| Panics                                                  | yes (event)                             | no (no panic hook)        |
| `tracing::error!` calls                                 | yes (event)                             | no                        |
| `tracing::warn!` calls                                  | yes (breadcrumb)                        | no                        |
| `tracing::info!`/`debug!` calls                         | yes (breadcrumb)                        | no                        |
| Performance transactions                                | only if `SENTRY_TRACES_SAMPLE_RATE > 0` | no                        |
| `POST /conversion`                                      | no                                      | yes (`conversion_completed`) |

### Privacy

`docspec-http` does NOT send the following to Sentry:

- Request bodies (markdown, HTML, or DOCX documents)
- Response bodies (BlockNote JSON, HTML, oxa.dev JSON, or Pandoc native)
- PII (Sentry default: `send_default_pii = false`)
- DSN values (never logged or echoed)

Sentry's default header redaction (Authorization, Cookie, etc.) is preserved.

Each captured event is tagged with `request_id` (UUID v4) and `trace_id`
(`X-Trace-ID` header value, if present) for correlation with logs.

### PostHog

`docspec-http` integrates with [PostHog](https://posthog.com/) for product analytics and error reporting.
Activation is fully opt-in via Cargo feature and environment variables — the binary has zero PostHog
overhead when the `posthog` feature is disabled or no API key is configured.

#### Activation

Enable the `posthog` Cargo feature and set ONE of the following:

- `DOCSPEC_POSTHOG_API_KEY` — docspec-specific override (preferred)
- `POSTHOG_API_KEY` — PostHog's standard convention (fallback)

**For `docspec-cli` users:** the power-user path is:

```bash
cargo install docspec-cli --features posthog
```

This enables PostHog analytics in the HTTP server without needing to specify the nested feature.

If both are set, `DOCSPEC_POSTHOG_API_KEY` wins. An empty string is treated as "not set" — the server
starts normally without PostHog.

#### Configuration (all optional)

- `DOCSPEC_POSTHOG_HOST` — preferred ingest host override
- `POSTHOG_HOST` — fallback ingest host (default: `https://us.i.posthog.com`)
- `POSTHOG_SAMPLE_RATE` — capture sample rate `[0.0, 1.0]` (default: `1.0`); set to `0.0` to disable all captures

#### What is captured

| Signal | Captured? | Event name |
| --- | --- | --- |
| `500 Internal Server Error` (`HttpError::Internal`) | yes | `$exception` |
| `422 Unprocessable Entity` (`HttpError::Unprocessable`) | yes | `$exception` |
| Other 4xx responses | no | — |
| Panics | no (no panic hook) | — |
| `POST /conversion` success or failure | yes | `conversion_completed` |

The `conversion_completed` event carries 10 properties: `result`, `error_class`, `input_mime_type`,
`output_mime_type`, `input_bytes`, `output_bytes`, `duration_ms`, `$app_version`, and optionally
`trace_id` and `$is_anonymous`.

#### Privacy

`docspec-http` does NOT send the following to PostHog:

- Request bodies (markdown, HTML, or DOCX documents)
- Response bodies (BlockNote JSON, HTML, oxa.dev JSON, or Pandoc native)
- PII
- Any property key containing "body" (stripped by `BeforeSendHook`)

Each error event uses a per-event anonymous UUID as `distinct_id` (with `$is_anonymous = true`).
Conversion events use the `X-Request-ID` header value when present, else a per-event UUID.

### PostHog + Sentry coexistence

When both `DOCSPEC_SENTRY_DSN` and `DOCSPEC_POSTHOG_API_KEY` are set (and both features are compiled
in), EVERY 500 and 422 response is reported to BOTH backends by design. This is the intentional
coexistence mode.

**Double-billing warning:** each error is counted as one event in Sentry and one event in PostHog.
To avoid double-billing, unset one of the two env vars OR compile without the corresponding feature
(see [Feature Flags](#feature-flags) below).

**Cross-tool correlation:** each PostHog error event carries a `sentry_event_id` property equal to
the Sentry `Uuid` for the same request, enabling one-click navigation from a PostHog funnel to the
corresponding Sentry issue.

## Wire Contract

Mirrors `github.com/docspecio/api` v3.0.2 where feasible: same endpoint path, RFC 7807 errors, `X-Request-ID`/`X-Trace-ID` header handling. Diverges in supported conversions.

## Graceful Shutdown

The server handles SIGINT and SIGTERM. In-flight requests complete before the process exits.

## Feature Flags

`docspec-http` exposes two optional Cargo features:

| Feature | Default | Description |
| --- | --- | --- |
| `sentry` | **on** (part of `default`) | Enables Sentry error reporting integration |
| `posthog` | off | Enables PostHog product analytics integration |

Default:

```toml
[dependencies]
docspec-http = "1"
```

Opt into PostHog:

```toml
[dependencies]
docspec-http = { version = "1", features = ["posthog"] }
```

Both backends:

```toml
[dependencies]
docspec-http = { version = "1", features = ["sentry", "posthog"] }
```

No telemetry:

```toml
[dependencies]
docspec-http = { version = "1", default-features = false }
```

> **Migration note:** if you depend on `docspec-http` with `default-features = false`, you will
> lose the Sentry integration after this release. Add `features = ["sentry"]` to preserve the
> previous behavior.

## Metrics

`docspec-http` exposes a Prometheus metrics endpoint on the same port as the main API.

**Endpoint**: `GET /metrics`

**Format**: Prometheus exposition format 0.0.4 (`text/plain; version=0.0.4; charset=utf-8`)

**Auth**: None. The endpoint is internal-only. See [Security](#security) below.

### Metric Catalog

| Name                                        | Type      | Labels                                                         | Description                                                                            | Buckets                                                                   |
| ------------------------------------------- | --------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| `docspec_http_requests_total`               | counter   | `method`, `path`, `status`                                     | Total HTTP requests received                                                           | —                                                                         |
| `docspec_http_request_duration_seconds`     | histogram | `method`, `path`, `status`                                     | HTTP request latency in seconds                                                        | 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0             |
| `docspec_http_request_body_bytes`           | histogram | `input_mime_type`                                              | HTTP request body size in bytes, labeled by input MIME type                            | 100, 200, 400, 800, 1600, 3200, 6400, 12800, 25600, 51200, 102400, 204800 |
| `docspec_conversions_total`                 | counter   | `result`, `error_class`, `input_mime_type`, `output_mime_type` | Total document conversions, labeled by result, error class, and input/output MIME type | —                                                                         |
| `docspec_conversion_duration_seconds`       | histogram | `result`, `input_mime_type`, `output_mime_type`                | Document conversion duration in seconds, labeled by result and input/output MIME type  | 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0             |
| **`docspec_conversion_output_bytes`** (NEW) | histogram | `input_mime_type`, `output_mime_type`                          | Document conversion output size in bytes (success only)                                | 100, 200, 400, 800, 1600, 3200, 6400, 12800, 25600, 51200, 102400, 204800 |

### Label Values

**`result`**: `success`, `client_error`, `server_error`

**`error_class`**: `body_not_utf8`, `empty_body`, `internal`, `method_not_allowed`, `not_acceptable`, `not_found`, `unprocessable`, `unsupported_media_type`, `none` (only when `result=success`)

**`input_mime_type`**: `text/markdown` (the request's Content-Type matched the markdown reader), `text/html` (the request's Content-Type matched the HTML reader), `application/vnd.openxmlformats-officedocument.wordprocessingml.document` (the request's Content-Type matched the DOCX reader), `unsupported` (Content-Type header present but not a supported input format), `none` (Content-Type header absent).

**`output_mime_type`**: `application/vnd.docspec.blocknote+json` (conversion succeeded; output produced by the BlockNote writer), `text/html` (conversion succeeded; output produced by the HTML writer), `application/vnd.oxa+json` (conversion succeeded; output produced by the oxa.dev writer), `application/vnd.pandoc.native` (conversion succeeded; output produced by the Pandoc native writer), `none` (no output produced — any error path).

**`path`**: matched route template (`/conversion`, `/health`) or `unknown` for fallback handlers

**`status`**: numeric HTTP status code as a string (e.g., `"200"`, `"422"`)

**`method`**: HTTP method as a string (e.g., `"GET"`, `"POST"`)

### Cardinality Guarantees

`path` is bounded to `{"/conversion", "/health", "unknown"}`. `error_class` is bounded to 9 values. `result` is bounded to 3 values. Per-request identifiers (`X-Request-ID`, `X-Trace-ID`) are never used as labels. `input_mime_type` is bounded to 5 values (`text/markdown`, `text/html`, `application/vnd.openxmlformats-officedocument.wordprocessingml.document`, `unsupported`, `none`). `output_mime_type` is bounded to 5 values (`application/vnd.docspec.blocknote+json`, `text/html`, `application/vnd.oxa+json`, `application/vnd.pandoc.native`, `none`). Both come from a fixed set of `&'static str` constants in the source — never from raw header values.

### Scrape Model

Each pod maintains its own in-memory metrics. Prometheus scrapes each pod independently. No inter-pod communication is required. Aggregate across pods using PromQL.

Upkeep runs every 5 seconds, keeping histogram internal state bounded.

The `/metrics` route is mounted outside the API middleware stack, so it does not include the global `Cache-Control` header used by API responses.

The body-size histogram (`docspec_http_request_body_bytes`) only records bodies that passed Content-Type and Accept validation. Rejected requests are not counted.

The output-bytes histogram (`docspec_conversion_output_bytes`) only records observations for successful conversions. Failed conversions do not produce output, so no observation is recorded.

### Example PromQL Queries

Per-pod request rate:

```promql
rate(docspec_http_requests_total[5m])
```

Aggregate p99 latency across all pods:

```promql
histogram_quantile(0.99, sum by (le) (rate(docspec_http_request_duration_seconds_bucket[5m])))
```

Error rate broken down by error class:

```promql
rate(docspec_conversions_total{result!="success"}[5m])
```

Body-size p95:

```promql
histogram_quantile(0.95, sum by (le) (rate(docspec_http_request_body_bytes_bucket[5m])))
```

Body-size p95 by input format:

```promql
histogram_quantile(0.95, sum by (le, input_mime_type) (rate(docspec_http_request_body_bytes_bucket[5m])))
```

Conversion success rate by input format:

```promql
sum by (input_mime_type) (rate(docspec_conversions_total{result="success"}[5m]))
  / sum by (input_mime_type) (rate(docspec_conversions_total[5m]))
```

### Security

`/metrics` has no authentication. It's intended for internal scraping only. Deploy behind a private overlay network or a Kubernetes `NetworkPolicy` that restricts access to your Prometheus pods.
