# Client SDK Specification

This document is the single reference for building a type-safe client library for rust-fafb-q in any language. It consolidates the [TCP wire protocol](protocol.md) and [HTTP API](api.md) into a client-implementer-focused spec with formal type definitions, state machines, and recommended API surface.

## Table of Contents

- [Type Definitions](#type-definitions)
- [Language Type Mappings](#language-type-mappings)
- [Connection State Machine](#connection-state-machine)
- [TCP Wire Protocol](#tcp-wire-protocol)
- [HTTP API](#http-api)
- [Recommended Client API Surface](#recommended-client-api-surface)
- [Error Handling](#error-handling)
- [Compatibility Testing](#compatibility-testing)
- [Implementation Checklist](#implementation-checklist)

---

## Type Definitions

All types a client SDK must define. Names are language-neutral — adapt casing to target language conventions (e.g. `MessageType` → `message_type` in Python, `MessageType` in Go).

### Scalar Types

| SDK Type | Wire Type | Size | Encoding |
|----------|-----------|------|----------|
| `MessageId` | `u64` | 8 bytes | Big-endian unsigned 64-bit integer |
| `TopicOffset` | `u64` | 8 bytes | Big-endian unsigned 64-bit integer |
| `ErrorCode` | `u16` | 2 bytes | Big-endian unsigned 16-bit integer |
| `ProtocolVersion` | `u8` | 1 byte | Unsigned 8-bit integer. Current value: `1` |
| `FrameLength` | `u32` | 4 bytes | Big-endian. Excludes the 4-byte length field itself |
| `PayloadLength` | `u32` | 4 bytes | Big-endian unsigned 32-bit integer |
| `StringLength` | `u16` | 2 bytes | Big-endian unsigned 16-bit integer |
| `Capabilities` | `u32` | 4 bytes | Bitmask of supported features. `CAP_GOING_AWAY = 0x01`, `CAP_RECONNECT = 0x02`, `CAP_NACK = 0x04`, `CAP_HEADERS = 0x08`, `CAP_WILDCARD = 0x10`. |

### String Types

All strings are UTF-8 encoded and length-prefixed with a 2-byte (`u16`) big-endian length.

| SDK Type | Constraints | Description |
|----------|-------------|-------------|
| `NamespaceId` | Non-empty UTF-8 string | Namespace identifier |
| `TopicName` | Non-empty UTF-8 string, max 255 bytes | Topic identifier within a namespace. On publish: must not contain `*` or `>`, no empty segments (`..`). On subscribe: `*` and `>` are allowed as wildcard pattern tokens (requires `CAP_WILDCARD`). |
| `ApiKey` | Non-empty UTF-8 string | Authentication credential |
| `GroupName` | UTF-8 string, may be empty | Consumer group name. Empty = fan-out subscriber |

### Enums

#### `MessageType`

```
Connect         = 0x01    client → server
ConnectAck      = 0x02    server → client
Subscribe       = 0x03    client → server
SubscribeAck    = 0x04    server → client
Unsubscribe     = 0x05    client → server
UnsubscribeAck  = 0x06    server → client
Publish         = 0x07    client → server
PublishAck      = 0x08    server → client
Deliver         = 0x09    server → client
Ack             = 0x0A    client → server
Error           = 0x0B    server → client
Ping            = 0x0C    either → either
Pong            = 0x0D    either → either
SubscribeAt     = 0x0E    client → server
SubscribeFrom   = 0x0F    client → server
GoingAway       = 0x10    server → client
Reconnect       = 0x11    client → server
ReconnectAck    = 0x12    server → client
Fetch           = 0x13    client → server
FetchResponse   = 0x14    server → client
FetchFrom       = 0x15    client → server
Nack            = 0x16    client → server
NackAck         = 0x17    server → client
```

Client SDKs should only construct **client → server** and **either → either** message types. Receiving an unexpected direction is a protocol error.

#### `ErrorCode`

| Value | Name | Fatal | Description |
|-------|------|-------|-------------|
| `0x0001` | `AuthFailed` | Yes | API key invalid or namespace not found |
| `0x0002` | `PermissionDenied` | No | Lacks required permission for operation |
| `0x0003` | `NamespaceNotFound` | No | Namespace does not exist |
| `0x0004` | `TopicNotFound` | No | Topic does not exist (auto-create disabled) |
| `0x0005` | `InvalidFrame` | Yes | Malformed frame, unknown type, or wrong direction |
| `0x0006` | `FrameTooLarge` | Yes | Frame length exceeds server `max_message_size` |
| `0x0007` | `UnsupportedVersion` | Yes | Protocol version not supported |
| `0x0008` | `NotAuthenticated` | Yes | Non-Connect message sent before authentication |
| `0x0009` | `InternalError` | No | Unexpected server-side error |
| `0x000A` | `ResourceExhausted` | No | Server resource limit reached (memory, connections) |
| `0x000B` | `SlowConsumer` | Yes | Client consuming too slowly; connection will be closed |
| `0x000C` | `OffsetNotAvailable` | No | Requested offset has been deleted by retention (informational, sent before SubscribeAck when offset is clamped) |
| `0x000D` | `ServerDraining` | No | Request rejected because the server is draining (planned shutdown). Subscribe/SubscribeAt/SubscribeFrom/Reconnect are rejected during drain. Publishes, Acks, and Fetch requests are still accepted. |
| `0x000E` | `MessageTooLarge` | No | First message at requested offset exceeds frame size budget (Fetch only). Skip with `offset + 1` or use the HTTP endpoint with larger budget (default 4MB). |
| `0x000F` | `InvalidTopic` | No | Topic name is invalid: contains wildcard characters on publish, exceeds 255 bytes, has empty segments, or wildcard pattern used without `CAP_WILDCARD`. |
| `0x0010` | `InvalidOperation` | No | Operation not supported in context: `SubscribeAt` with wildcard pattern, or unsubscribing a specific topic within a wildcard subscription. |

**Fatal** errors mean the server will close the connection after sending the Error frame. Client SDKs must treat the connection as dead after receiving a fatal error — do not attempt to send further messages.

#### `Permission`

```
Subscribe = "subscribe"
Publish   = "publish"
Query     = "query"
Fetch     = "fetch"
```

Permissions are configured server-side per API key. The client does not transmit permissions — they are enforced by the server. Client SDKs should expose these as an enum for documentation and error-handling purposes.

### Composite Types

#### `Frame`

A discriminated union (tagged union / sum type / algebraic data type) over `MessageType`:

| Variant | Fields |
|---------|--------|
| `Connect` | `version: ProtocolVersion`, `namespace: NamespaceId`, `api_key: ApiKey`, `capabilities: Capabilities?` |
| `ConnectAck` | `success: bool`, `message: string`, `server_capabilities: Capabilities?` |
| `Subscribe` | `topic: TopicName`, `group: GroupName?` |
| `SubscribeAck` | `topic: TopicName`, `start_offset: TopicOffset`, `group: GroupName?`, `matched_count: u16?` |
| `Unsubscribe` | `topic: TopicName`, `group: GroupName?` |
| `UnsubscribeAck` | `topic: TopicName`, `group: GroupName?` |
| `Publish` | `topic: TopicName`, `ttl_ms: u32?`, `headers: Header[]?`, `payload: bytes` |
| `PublishAck` | `message_id: MessageId` |
| `Deliver` | `message_id: MessageId`, `offset: TopicOffset`, `timestamp_millis: u64`, `ttl_ms: u32?`, `topic: TopicName`, `headers: Header[]?`, `payload: bytes` |
| `Ack` | `message_id: MessageId` |
| `Error` | `code: ErrorCode`, `message: string` |
| `Ping` | *(no fields)* |
| `Pong` | *(no fields)* |
| `SubscribeAt` | `topic: TopicName`, `offset: TopicOffset`, `group: GroupName?` |
| `SubscribeFrom` | `topic: TopicName`, `timestamp_millis: u64`, `group: GroupName?` |
| `GoingAway` | `reason: GoingAwayReason`, `deadline_ms: u32`, `message: string` |
| `Reconnect` | `version: ProtocolVersion`, `namespace: NamespaceId`, `api_key: ApiKey`, `capabilities: Capabilities`, `subscriptions: SubscriptionEntry[]` |
| `ReconnectAck` | `status: ReconnectStatus`, `message: string`, `server_capabilities: Capabilities`, `results: SubscriptionResult[]` |
| `Fetch` | `topic: TopicName`, `offset: TopicOffset`, `max_count: u32` |
| `FetchResponse` | `topic: TopicName`, `next_offset: TopicOffset`, `head_offset: TopicOffset`, `messages: FetchedMessage[]` |
| `FetchFrom` | `topic: TopicName`, `timestamp_millis: u64`, `max_count: u32` |
| `Nack` | `message_id: MessageId`, `reason: string` |
| `NackAck` | `message_id: MessageId`, `status: NackAckStatus` |

The `group` field is **optional**: on the wire, an empty string (length 0) represents "no group" (fan-out mode). Client SDKs should represent this as `null`/`None`/`undefined` rather than exposing the empty-string encoding.

#### `TopicInfo` (HTTP response)

```
namespace:            string
topic:                string
depth:                integer    // messages currently buffered
bytes_used:           integer    // total payload bytes in buffer
base_offset:          integer    // lowest retained offset
head_offset:          integer    // next offset to be assigned
subscriber_count:     integer    // TCP subscribers only
consumer_group_count: integer    // TCP consumer groups only
```

#### `PublishResponse` (HTTP response)

```
offset:           integer    // sequential offset within topic
message_id:       integer    // unique message identifier
timestamp_millis: integer    // publish timestamp (milliseconds since epoch)
```

#### `TopicListResponse` (HTTP response)

```
namespace: string
topics:    string[]
```

#### `GoingAwayReason`

A newtype wrapping `u8`. Unknown values must be accepted (not rejected) for forward compatibility.

| Value | Constant | Description |
|-------|----------|-------------|
| `0x01` | `UPGRADE` | Server upgrading |
| `0x02` | `MAINTENANCE` | Server maintenance |
| `0x03` | `OVERLOADED` | Server shedding load |
| `0x04` | `SHUTDOWN` | Server shutting down permanently |

#### `ReconnectStatus`

| Value | Name | Description |
|-------|------|-------------|
| `0x00` | `Success` | All subscriptions restored (OffsetClamped counts as success) |
| `0x01` | `Partial` | Some subscriptions failed — all rolled back, no active subscriptions |
| `0x02` | `AuthFailed` | Authentication failed — connection closed |

#### `SubscriptionStatus`

| Value | Name | Hard Failure | Description |
|-------|------|-------------|-------------|
| `0x00` | `Restored` | No | Subscription restored at requested offset |
| `0x01` | `OffsetClamped` | No (warning) | Offset clamped to oldest available |
| `0x02` | `TopicNotFound` | Yes | Topic does not exist |
| `0x03` | `PermissionDenied` | Yes | Client lacks Subscribe permission |
| `0x04` | `GroupFull` | Yes | Consumer group at capacity |

#### `NackAckStatus`

| Value | Name | Description |
|-------|------|-------------|
| `0x00` | `DeadLettered` | Message routed to DLQ topic |
| `0x01` | `Dropped` | DLQ not enabled — message removed from inflight |
| `0x02` | `WriteFailed` | DLQ write failed — message remains inflight for redelivery |

#### `SubscriptionEntry` (Reconnect request)

```
topic:             TopicName
group:             GroupName?
last_acked_offset: TopicOffset      // server resumes from last_acked_offset + 1
```

#### `SubscriptionResult` (ReconnectAck response)

```
topic:                TopicName
group:                GroupName?
status:               SubscriptionStatus
actual_start_offset:  TopicOffset    // where delivery will actually start
```

#### `Header`

A single message header — key-value pair of byte strings. Both key and value are opaque bytes on the wire (typically UTF-8 keys, but not enforced). Duplicate keys are allowed. Insertion order is preserved.

```
key:   bytes   // header key (max 256 bytes)
value: bytes   // header value (max 8 KB)
```

`Headers` is an ordered list of `Header` entries: `Header[]`.

#### `FetchedMessage`

A single message within a `FetchResponse`. Lighter than `Deliver` — no `MessageId` (fetch consumers don't ack).

```
offset:           TopicOffset   // position within topic log
timestamp_millis: u64           // publish timestamp (milliseconds since epoch)
ttl_ms:           u32           // original TTL in milliseconds (0 = no expiry; only present when CAP_MESSAGE_TTL negotiated)
headers:          Header[]      // message headers (empty if none; only present when CAP_HEADERS negotiated)
payload:          bytes         // raw message payload
```

#### `FetchBatch` (recommended SDK wrapper)

```
topic:       TopicName
next_offset: TopicOffset      // use as `offset` in next fetch() call
head_offset: TopicOffset      // topic's current head (next to be assigned)
messages:    FetchedMessage[]  // 0 to max_count messages
has_more:    boolean           // convenience: next_offset < head_offset
```

`has_more` is computed client-side from `next_offset < head_offset`. This is reliable because `head_offset` is included in every `FetchResponse`.

#### `ErrorResponse` (HTTP response)

```
error:
  code:    integer    // numeric error code (shared with TCP)
  name:    string     // machine-readable name (e.g. "AUTH_FAILED")
  message: string     // human-readable description
```

---

## Language Type Mappings

### Wire primitives to language types

| Wire Type | TypeScript | Python | Go | Java | C# |
|-----------|-----------|--------|-----|------|-----|
| `u8` | `number` | `int` | `uint8` | `byte` (unsigned) | `byte` |
| `u16` | `number` | `int` | `uint16` | `int` (widen to avoid sign) | `ushort` |
| `u32` | `number` | `int` | `uint32` | `long` (widen to avoid sign) | `uint` |
| `u64` | `bigint` | `int` | `uint64` | `long` (note: signed, but sufficient range) | `ulong` |
| `string` | `string` | `str` | `string` | `String` | `string` |
| `bytes` | `Buffer` / `Uint8Array` | `bytes` | `[]byte` | `byte[]` | `byte[]` |
| `bool` | `boolean` | `bool` | `bool` | `boolean` | `bool` |

### Enum representation

| Language | Recommended Pattern |
|----------|-------------------|
| TypeScript | `const enum` or string union for `ErrorCode`, `MessageType` |
| Python | `enum.IntEnum` for wire codes, `enum.Enum` for `Permission` |
| Go | `const` block with typed `uint8`/`uint16` |
| Java | `enum` with `value` field |
| C# | `enum : byte` / `enum : ushort` |

### Discriminated union / Frame representation

| Language | Recommended Pattern |
|----------|-------------------|
| TypeScript | Discriminated union: `type Frame = ConnectFrame \| SubscribeFrame \| ...` with `type` field |
| Python | `dataclass` per variant + `Union` type alias, or `match` statement (3.10+) |
| Go | Interface `Frame` with per-variant structs implementing it |
| Java | Sealed interface `Frame` with record subclasses (17+) |
| C# | Abstract record `Frame` with derived records |

---

## Connection State Machine

Client SDKs must enforce this state machine. Attempting to send a message that is invalid for the current state should raise a client-side error **before** writing to the wire.

```
                    ┌─────────────────┐
        TCP connect │  Disconnected   │
           ┌────────┤                 │
           │        └─────────────────┘
           │          ▲   ▲       ▲
           │          │   │       │
           │     TCP close│  ReconnectAck(AuthFailed)
           │          │   │       │
           │          │  fatal Error / ConnectAck(failure)
           ▼          │   │
    ┌─────────────────┐   │
    │  Connected       │──┘
    │  (unauthenticated)│
    │                  │
    │  valid sends:    │
    │   - Connect      │
    │   - Reconnect    │
    └────────┬─────────┘
             │
             │ ConnectAck(success) or ReconnectAck(Success/Partial)
             ▼
    ┌─────────────────┐
    │  Authenticated   │◄──────────────────────────┐
    │                  │                            │
    │  valid sends:    │   UnsubscribeAck /         │
    │   - Subscribe    │   all topics unsubscribed  │
    │   - SubscribeAt  │                            │
    │   - SubscribeFrom│                            │
    │   - Publish      │                            │
    │   - Fetch        │                            │
    │   - FetchFrom    │                            │
    │   - Ping         ├───────────────────────┐    │
    └────────┬─────────┘                       │    │
             │                                 │    │
             │ SubscribeAck                    │    │
             ▼                                 │    │
    ┌─────────────────┐                        │    │
    │  Subscribed      │───────────────────────►┘   │
    │                  │                            │
    │  valid sends:    │                            │
    │   - Subscribe    │ (additional topics)        │
    │   - SubscribeAt  │                            │
    │   - SubscribeFrom│                            │
    │   - Unsubscribe  │────────────────────────────┘
    │   - Publish      │
    │   - Ack          │
    │   - Nack         │
    │   - Fetch        │
    │   - FetchFrom    │
    │   - Ping         │
    └────────┬─────────┘
             │
             │ GoingAway received
             ▼
    ┌─────────────────┐
    │  Draining        │
    │                  │
    │  valid sends:    │
    │   - Publish      │
    │   - Ack          │
    │   - Nack         │
    │   - Unsubscribe  │
    │   - Fetch        │
    │   - FetchFrom    │
    │   - Ping         │
    │                  │
    │  rejected:       │
    │   - Subscribe*   │ → ServerDraining
    └──────────────────┘
             │
             │ disconnect / force-close
             ▼
        Disconnected
```

Note: GoingAway is also possible from `Authenticated` state (transitions to `Draining` from either `Authenticated` or `Subscribed`).

### State transitions

| Current State | Event | Next State |
|---------------|-------|------------|
| `Disconnected` | TCP connection established | `Connected` |
| `Connected` | Send `Connect` → receive `ConnectAck(success)` | `Authenticated` |
| `Connected` | Send `Reconnect` → receive `ReconnectAck(Success)` | `Subscribed` (if subscriptions restored) or `Authenticated` |
| `Connected` | Send `Reconnect` → receive `ReconnectAck(Partial)` | `Authenticated` (no active subscriptions — atomic rollback) |
| `Connected` | Receive `ConnectAck(failure)` or `ReconnectAck(AuthFailed)` | `Disconnected` |
| `Connected` | Receive fatal `Error` | `Disconnected` |
| `Authenticated` | Receive `SubscribeAck` | `Subscribed` |
| `Subscribed` | Receive `UnsubscribeAck` and no active subscriptions remain | `Authenticated` |
| `Subscribed` | Receive `SubscribeAck` (additional topic) | `Subscribed` |
| `Authenticated` or `Subscribed` | Receive `GoingAway` | `Draining` |
| `Draining` | Client disconnects or server force-closes | `Disconnected` |
| Any | Receive fatal `Error` | `Disconnected` |
| Any | TCP connection closed | `Disconnected` |

### Valid messages per state

| State | Client may send | Client may receive |
|-------|----------------|--------------------|
| `Disconnected` | *(nothing)* | *(nothing)* |
| `Connected` | `Connect`, `Reconnect` | `ConnectAck`, `ReconnectAck`, `Error` |
| `Authenticated` | `Subscribe`, `SubscribeAt`, `SubscribeFrom`, `Publish`, `Fetch`, `FetchFrom`, `Ping` | `SubscribeAck`, `PublishAck`, `FetchResponse`, `GoingAway`, `Error`, `Ping`, `Pong` |
| `Subscribed` | `Subscribe`, `SubscribeAt`, `SubscribeFrom`, `Unsubscribe`, `Publish`, `Ack`, `Nack`, `Fetch`, `FetchFrom`, `Ping` | `SubscribeAck`, `UnsubscribeAck`, `PublishAck`, `Deliver`, `NackAck`, `FetchResponse`, `GoingAway`, `Error`, `Ping`, `Pong` |
| `Draining` | `Publish`, `Ack`, `Nack`, `Unsubscribe`, `Fetch`, `FetchFrom`, `Ping` | `PublishAck`, `UnsubscribeAck`, `Deliver`, `NackAck`, `FetchResponse`, `Error`, `Ping`, `Pong` |

---

## TCP Wire Protocol

### Frame encoding

Every message on the wire uses this envelope:

```
┌──────────────────┬───────────────┬──────────────────┐
│ Frame Length (4B) │ Msg Type (1B) │ Payload (N bytes) │
└──────────────────┴───────────────┴──────────────────┘
```

- **Frame Length** (`u32` BE): byte count of everything after this field (message type + payload).
- **Message Type** (`u8`): one of the `MessageType` enum values.
- **Payload**: message-specific fields laid out sequentially with no padding or alignment.

### String encoding

```
┌────────────────────┬──────────────────┐
│ String Length (2B)  │ UTF-8 Data (N B) │
└────────────────────┴──────────────────┘
```

- `u16` BE length prefix, then raw UTF-8 bytes.
- Length 0 = empty string. Used for absent optional fields (e.g. no consumer group).

### Payload encoding (Publish / Deliver)

```
┌─────────────────────────┬──────────────────┐
│ Payload Length (4B)      │ Raw Data (N B)   │
└─────────────────────────┴──────────────────┘
```

- `u32` BE length prefix (not `u16` like strings) to support payloads up to max frame size.

### Message field layouts

#### Connect (0x01) — client → server

| Field | Encoding |
|-------|----------|
| `version` | `u8` — must be `1` |
| `namespace` | length-prefixed string |
| `api_key` | length-prefixed string |
| `capabilities` | `u32` BE — **optional.** Bitmask of client capabilities. If frame has no remaining bytes after `api_key`, defaults to `0` (legacy v1 client). |

#### ConnectAck (0x02) — server → client

| Field | Encoding |
|-------|----------|
| `status` | `u8` — `0x00` success, `0x01` failure |
| `message` | length-prefixed string — empty on success |
| `server_capabilities` | `u32` BE — **optional.** Negotiated capabilities (`client_caps & SERVER_CAPABILITIES`). Present only when client sent capabilities. |

#### Subscribe (0x03) — client → server

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty) |
| `group` | length-prefixed string — empty = fan-out, non-empty = consumer group |

#### SubscribeAck (0x04) — server → client

Sent in response to `Subscribe`, `SubscribeAt`, or `SubscribeFrom`.

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string — echoes the pattern for wildcard subscriptions |
| `start_offset` | `u64` BE — actual starting offset (may differ from requested due to clamping). 0 for wildcard subscriptions. |
| `group` | length-prefixed string |
| `matched_count` | `u16` BE — always encoded by the current server. Number of existing topics matching the subscription (0 for exact-match). Decoders should check for 2+ remaining bytes after `group` — if present, read as `u16`; if absent (older server), default to 0. Old decoders that stop reading after `group` safely ignore trailing bytes. |

#### Unsubscribe (0x05) — client → server

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string |
| `group` | length-prefixed string — must match original subscription |

#### UnsubscribeAck (0x06) — server → client

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string |
| `group` | length-prefixed string |

#### Publish (0x07) — client → server

**Without `CAP_HEADERS`:**

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty) |
| `payload` | `u32` BE length + raw bytes |

**With `CAP_HEADERS` negotiated:**

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty) |
| `header_count` | `u16` BE — number of header entries (may be 0) |
| `headers` | `header_count` × `(key_len: u16 BE, key: bytes, value_len: u32 BE, value: bytes)` |
| `payload` | `u32` BE length + raw bytes |

Header keys starting with `_fafb.` are reserved and rejected by the server.

#### PublishAck (0x08) — server → client

| Field | Encoding |
|-------|----------|
| `message_id` | `u64` BE |

#### Deliver (0x09) — server → client

**Base format:**

| Field | Encoding |
|-------|----------|
| `message_id` | `u64` BE |
| `offset` | `u64` BE — position within the topic log |
| `timestamp_millis` | `u64` BE — publish timestamp (milliseconds since epoch) |
| `ttl_ms` | `u32` BE — *(only with `CAP_MESSAGE_TTL`)* original TTL in ms, 0 = no expiry. Compute remaining: `(timestamp_millis + ttl_ms) - now_ms`. |
| `topic` | length-prefixed string |
| `header_count` | `u16` BE — *(only with `CAP_HEADERS`)* number of header entries (may be 0) |
| `headers` | *(only with `CAP_HEADERS`)* `header_count` × `(key_len: u16 BE, key: bytes, value_len: u32 BE, value: bytes)` |
| `payload` | `u32` BE length + raw bytes |

Optional fields (`ttl_ms`, `header_count`/`headers`) are only present on the wire when the corresponding capability is negotiated. Without `CAP_MESSAGE_TTL`, the TTL field is omitted entirely (not sent as 0). Without `CAP_HEADERS`, the header section is omitted.

#### Ack (0x0A) — client → server

| Field | Encoding |
|-------|----------|
| `message_id` | `u64` BE |

#### Error (0x0B) — server → client

| Field | Encoding |
|-------|----------|
| `code` | `u16` BE |
| `message` | length-prefixed string |

#### Ping (0x0C) — either → either

Empty payload.

#### Pong (0x0D) — either → either

Empty payload.

#### SubscribeAt (0x0E) — client → server

Subscribe starting from a specific offset. Requires Subscribe permission. **Not supported with wildcard patterns** — returns `ErrorCode::InvalidOperation` (offsets are per-topic, meaningless across topics).

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty, must not contain wildcards) |
| `offset` | `u64` BE — the offset to start consuming from |
| `group` | length-prefixed string — empty = fan-out, non-empty = consumer group |

The server responds with `SubscribeAck`. The `start_offset` in the ack may differ from the requested offset if it was clamped (past head or below oldest retained data). When clamping occurs, the server sends a non-fatal `OffsetNotAvailable` error frame before the `SubscribeAck`.

#### SubscribeFrom (0x0F) — client → server

Subscribe starting from the first message at or after a timestamp. Requires Subscribe permission. Supports wildcard patterns when `CAP_WILDCARD` is negotiated — the timestamp is resolved independently for each matching topic.

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty, may contain wildcard patterns with `CAP_WILDCARD`) |
| `timestamp_millis` | `u64` BE — milliseconds since Unix epoch |
| `group` | length-prefixed string — empty = fan-out, non-empty = consumer group |

The server resolves the timestamp to an offset using the segment index, then behaves like `SubscribeAt`. If no message exists at or after the timestamp, the subscription starts at the current head (live tail). The server responds with `SubscribeAck`. For wildcard subscriptions, `matched_count` in the ack indicates how many topics matched.

#### GoingAway (0x10) — server → client

Server-initiated planned shutdown notification. Only sent to clients that negotiated `CAP_GOING_AWAY`. See [client_sdk_deploy.md](client_sdk_deploy.md) for handling guidance.

| Field | Encoding |
|-------|----------|
| `reason` | `u8` — reason code (see `GoingAwayReason`). Unknown values must be accepted. |
| `deadline_ms` | `u32` BE — milliseconds until server may force-close. Hint, not a contract. |
| `message` | length-prefixed string — human-readable message |

#### Reconnect (0x11) — client → server

Batch re-authenticate and re-subscribe in one round-trip. Must be sent in pre-auth state. Requires `CAP_RECONNECT`.

| Field | Encoding |
|-------|----------|
| `version` | `u8` — must be `1` |
| `namespace` | length-prefixed string |
| `api_key` | length-prefixed string |
| `capabilities` | `u32` BE — client capabilities bitmask |
| `subscription_count` | `u16` BE — number of subscription entries (max 10,000) |
| `subscriptions` | `subscription_count` × `(topic: string, group: string, last_acked_offset: u64 BE)` |

The server authenticates, negotiates capabilities, then attempts to restore all subscriptions atomically. Responds with `ReconnectAck`.

#### ReconnectAck (0x12) — server → client

Response to `Reconnect` with overall status and per-subscription results.

| Field | Encoding |
|-------|----------|
| `status` | `u8` — `0x00` Success, `0x01` Partial, `0x02` AuthFailed (see `ReconnectStatus`) |
| `message` | length-prefixed string |
| `server_capabilities` | `u32` BE — negotiated capabilities |
| `result_count` | `u16` BE — number of subscription results |
| `results` | `result_count` × `(topic: string, group: string, sub_status: u8, actual_start_offset: u64 BE)` |

On `Success`: all subscriptions active, delivery begins. On `Partial`: all subscriptions rolled back (zero active), client should fall back to `Connect + SubscribeAt`. On `AuthFailed`: connection closed.

#### Fetch (0x13) — client → server

Stateless pull-based message retrieval. Does not create subscriptions, delivery tasks, or inflight tracking. Requires `Fetch` permission (independent of `Subscribe`).

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty) |
| `offset` | `u64` BE — starting offset (inclusive). If below oldest retained offset, server clamps and sends `OffsetNotAvailable` warning before `FetchResponse`. |
| `max_count` | `u32` BE — maximum messages to return. Must be > 0 (server returns `InvalidFrame` if zero). Server may return fewer due to frame size limits or end of topic. |

#### FetchResponse (0x14) — server → client

Sent in response to both `Fetch` and `FetchFrom`.

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string — echoes requested topic |
| `next_offset` | `u64` BE — offset for next Fetch call. If `count > 0`: last returned offset + 1. If `count == 0`: topic's head offset. |
| `head_offset` | `u64` BE — topic's current head offset (next offset to be assigned). Use `next_offset < head_offset` to check if more messages are available. |
| `count` | `u32` BE — number of message entries following |
| `messages` | `count` × message entry (see below) |

**Message entry (without `CAP_HEADERS`):** `(offset: u64 BE, timestamp_millis: u64 BE, payload_len: u32 BE, payload: bytes)`

**Message entry (with `CAP_HEADERS`):** `(offset: u64 BE, timestamp_millis: u64 BE, header_count: u16 BE, headers: header_count × (key_len: u16 BE, key: bytes, value_len: u32 BE, value: bytes), payload_len: u32 BE, payload: bytes)`

> **Frame size and batch truncation:** The entire `FetchResponse` frame must fit within the server's `max_message_size` (default 1MB). If the requested `max_count` would produce a response exceeding this limit, the server returns fewer messages. Use `next_offset < head_offset` to detect that more messages are available, then issue another Fetch with `offset = next_offset`. For topics with large messages (e.g., 500KB payloads), a single Fetch may return only 1 message.

> **Oversized messages (`MessageTooLarge`):** If the message at the requested offset is individually larger than the frame size limit, the server returns `Error(MessageTooLarge, 0x000E)` with the stuck offset and message size. The client should skip (`offset + 1`), request the operator increase `max_message_size`, or use the HTTP fetch endpoint which has a separate, larger size budget (default 4MB).

#### FetchFrom (0x15) — client → server

Pull messages starting from a timestamp. The server resolves the timestamp to an offset internally and responds with a `FetchResponse` (same frame as `Fetch`). Requires only `Fetch` permission — no `Subscribe` permission needed.

| Field | Encoding |
|-------|----------|
| `topic` | length-prefixed string (non-empty) |
| `timestamp_millis` | `u64` BE — fetch from first message at or after this timestamp (milliseconds since epoch). If no message exists at or after this timestamp, returns empty response at head. |
| `max_count` | `u32` BE — maximum messages to return. Must be > 0. |

#### Nack (0x16) — client → server

Explicitly reject a delivered message. Requires `CAP_NACK` capability.

| Field | Encoding |
|-------|----------|
| `message_id` | `u64` BE — the message ID being rejected (from Deliver) |
| `reason` | length-prefixed string — human-readable rejection reason (max 1024 bytes, truncated if exceeded) |

Server responds with `NackAck`. If DLQ is enabled, the message is routed to the dead-letter queue. Unlike Ack (fire-and-forget), Nack always returns a response.

#### NackAck (0x17) — server → client

Response to a `Nack` frame.

| Field | Encoding |
|-------|----------|
| `message_id` | `u64` BE — echoes the message ID from the Nack |
| `status` | `u8` — `0x00` DeadLettered, `0x01` Dropped, `0x02` WriteFailed (see `NackAckStatus`) |

> **Note for SDK developers:** `Fetch` and `FetchFrom` do NOT change the connection state. They are stateless request/response operations that can be freely interleaved with Subscribe/Publish on the same connection. Fetch requests on a single connection are processed sequentially — for parallel fetches from multiple topics, use multiple connections or the HTTP endpoint.

**Consumer group semantics:** When creating a new group, the group starts at the requested offset/timestamp. When joining an existing group, the new member joins at the group's current cursor position regardless of the requested offset — use a new group name to replay from a specific position. The `SubscribeAck.start_offset` will reflect the group's current cursor (not the requested offset). If the requested offset differs from the group's cursor, an `OffsetNotAvailable` (0x000C) warning frame may precede the `SubscribeAck`. Client SDKs should check `start_offset` in the ack to determine the actual starting position and handle the informational warning gracefully.

### Limits

| Parameter | Default | Configurable |
|-----------|---------|--------------|
| Max frame size | 1,048,576 bytes (1 MB) | `tcp.max_message_size` |
| Min frame length | 1 byte (message type only) | No |
| Max string length | 65,535 bytes (u16 max) | No |
| Protocol version | `1` | No |

---

## HTTP API

Base URL: `http://{host}:{port}` (default `http://127.0.0.1:8080`).

All endpoints except `/health/*` require `Authorization: Bearer <api_key>`. Admin endpoints (`/admin/*`) use a separate `admin_api_key`.

### Endpoints

#### `GET /health/live`

No authentication. Returns `200 OK` with `{ "status": "alive" }`. Always available.

#### `GET /health/ready`

No authentication. Returns `200 OK` with `{ "status": "ready" }` when initialized and running. Returns `503` during initialization, drain, or shutdown.

#### `GET /health`

Alias for `/health/ready`.

#### `POST /v1/{namespace}/{topic}/publish`

Publish a message. Topic auto-created on first publish.

- **Permission required**: `Publish`
- **Request body**: raw bytes (the payload)
- **Message headers**: Pass via `X-Fafb-*` HTTP headers (prefix stripped to form message header key). Same validation limits as TCP. Reserved prefix `_fafb.` rejected with 400.
- **Response** (`200`): `PublishResponse` JSON
- **Errors**: `401`, `403`, `404`, `503` (queue full)

#### `GET /v1/{namespace}/{topic}/info`

Get topic statistics.

- **Permission required**: `Query`
- **Response** (`200`): `TopicInfo` JSON
- **Errors**: `401`, `403`, `404`

#### `GET /v1/{namespace}/topics`

List topics in a namespace.

- **Permission required**: `Query`
- **Response** (`200`): `TopicListResponse` JSON
- **Errors**: `401`, `403`, `404`

#### `GET /v1/{namespace}/{topic}/fetch`

Pull a batch of messages. Stateless — no subscriptions or server-side state created.

- **Permission required**: `Fetch`
- **Query parameters**: `offset` (u64) OR `timestamp` (u64) — mutually exclusive. `max_count` (optional, default 100, max 10,000).
- **Response** (`200`): JSON with `topic`, `next_offset`, `head_offset`, `messages[]` (each with `offset`, `timestamp_millis`, optional `headers[]`, `payload_base64`)
- **Optional field**: `warning` string — present when requested offset was clamped by retention
- **Errors**: `400` (invalid params), `401`, `403`, `404`, `413` (message too large)

> **Important:** HTTP payloads are **base64-encoded** in the `payload_base64` field. SDKs must decode before returning to users. This differs from TCP where payloads are raw bytes.

> **HTTP vs TCP budget:** The HTTP endpoint uses `http.max_fetch_response_bytes` (default 4MB) vs the TCP frame limit (default 1MB). Messages that return `MessageTooLarge` over TCP may be fetchable via HTTP. SDKs should document this fallback.

> **`max_count` limit:** The HTTP endpoint enforces `max_count <= 10,000` (returning 400). TCP has no upper bound on `max_count` — the server truncates by frame size only. SDKs should document this difference.

> **Offset clamping:** TCP sends a separate `OffsetNotAvailable` error frame before `FetchResponse`. HTTP flattens this into a `"warning"` field on the JSON response body. SDKs should handle both patterns.

See [api.md](api.md) for full endpoint documentation.

### HTTP error mapping

| ErrorCode | HTTP Status |
|-----------|-------------|
| `AuthFailed` / `NotAuthenticated` | `401 Unauthorized` |
| `PermissionDenied` | `403 Forbidden` |
| `NamespaceNotFound` / `TopicNotFound` | `404 Not Found` |
| `FrameTooLarge` | `413 Payload Too Large` |
| `ResourceExhausted` | `503 Service Unavailable` |
| `MessageTooLarge` | `413 Payload Too Large` |
| `InternalError` | `500 Internal Server Error` |

---

## Recommended Client API Surface

This section defines the public interface a client SDK should expose. Adapt naming conventions to target language idioms.

### Constructor / Configuration

```
Client(options: ClientOptions) → Client

ClientOptions:
  host:       string              // server hostname
  port:       number              // TCP port (default: 9090) or HTTP port (default: 8080)
  namespace:  NamespaceId         // target namespace
  api_key:    ApiKey              // authentication credential
  transport:  "tcp" | "http"      // which transport to use (default: "tcp")
  tls:        TlsOptions?        // optional, for future TLS support
  tcp:        TcpOptions?        // TCP-specific options
  http:       HttpOptions?       // HTTP-specific options

TcpOptions:
  auto_reconnect:        boolean   // reconnect on disconnect (default: true)
  reconnect_interval_ms: number    // backoff base (default: 1000)
  max_reconnect_attempts: number   // 0 = unlimited (default: 0)
  ping_interval_ms:      number    // keepalive interval (default: 10000)
  ping_timeout_ms:       number    // pong deadline (default: 10000)

#### Keepalive Contract

The server enforces an idle timeout (default 30 seconds). Clients MUST send `Ping` frames at a regular interval when the connection is idle to prevent disconnection. The recommended `ping_interval_ms` is 10000 (10 seconds) — at most one-third of the server's idle timeout.

- Any outbound frame (Publish, Ack, Subscribe, etc.) resets the keepalive timer — only send Ping when idle
- If the server sends a `Ping`, the client MUST reply with `Pong` immediately
- Failure to maintain keepalive results in `Error { InternalError, "idle timeout" }` and connection close
- Keepalive continues during drain (after GoingAway is received)

HttpOptions:
  base_url:   string?             // override full base URL
  timeout_ms: number              // request timeout (default: 30000)
```

### Connection lifecycle (TCP only)

```
client.connect()    → Promise<void>       // TCP connect + authenticate
client.disconnect() → Promise<void>       // graceful close
client.isConnected  → boolean             // current state
```

### Publish

```
// TCP transport: sends Publish frame, waits for PublishAck
// HTTP transport: POST /v1/{namespace}/{topic}/publish
client.publish(topic: TopicName, payload: bytes, headers?: Header[]) → Promise<PublishResult>

PublishResult:
  message_id: MessageId
  offset:     TopicOffset         // only from HTTP; TCP PublishAck has message_id only
```

### Subscribe (TCP only)

```
// Subscribe at current head (live tail)
// topic may be a wildcard pattern (e.g., "orders.*") when CAP_WILDCARD is negotiated
client.subscribe(topic: TopicName, group?: GroupName) → Promise<Subscription>

// Subscribe starting from a specific offset — NOT supported with wildcard patterns
client.subscribeAt(topic: TopicName, offset: TopicOffset, group?: GroupName) → Promise<Subscription>

// Subscribe starting from a timestamp (first message at or after)
// Supports wildcard patterns — timestamp resolved per matching topic
client.subscribeFrom(topic: TopicName, timestamp: u64, group?: GroupName) → Promise<Subscription>

Subscription:
  topic:         TopicName       // echoes pattern for wildcard subscriptions
  group:         GroupName?
  start_offset:  TopicOffset     // actual starting offset (0 for wildcard subscriptions)
  matched_count: u16             // topics currently matching (0 for exact match)
  unsubscribe:   () → Promise<void>
```

All three methods send their respective frame type and await `SubscribeAck`. The `start_offset` in the subscription object comes from the ack and should be checked by the caller — it may differ from the requested offset if the data was deleted by retention.

**Wildcard subscriptions** (requires `CAP_WILDCARD`): Pass a pattern like `"orders.*"` or `"events.>"` as the topic. The server matches all existing topics and delivers from them via the same channel. New topics created later that match the pattern are automatically added. Consumer groups work with wildcards — per-topic round-robin is maintained.

**Wildcard limitations:**
- `subscribeAt()` rejects wildcard patterns with `InvalidOperation` — offsets are per-topic
- Unsubscribe must use the original pattern, not a specific matched topic
- `matched_count` in the subscription indicates how many topics matched at subscribe time; more may be added via auto-subscribe

### Message delivery (TCP only)

```
// Register a handler for delivered messages
client.onDeliver(handler: (message: DeliveredMessage) → void)

DeliveredMessage:
  message_id:      MessageId
  offset:          TopicOffset   // position within topic log
  timestamp_millis: u64          // publish timestamp (millis since epoch)
  topic:           TopicName
  headers:         Header[]      // message headers (empty if none)
  payload:         bytes
  ack:             () → Promise<void>              // sends Ack frame for this message_id
  nack:            (reason?: string) → Promise<NackAckStatus>  // sends Nack frame (requires CAP_NACK)
```

The `ack()` and `nack()` convenience methods on each delivered message are the recommended pattern — they prevent mismatched message IDs and make the per-message flow explicit. `nack()` requires `CAP_NACK` to have been negotiated — SDKs should raise a client-side error if called without the capability.

### Fetch (pull consumption — TCP or HTTP)

**Wildcard restriction:** Fetch does not support wildcard patterns. The topic must be a concrete topic name. Wildcard patterns return `ErrorCode::InvalidTopic`. Use Subscribe for wildcard consumption, or fetch each topic individually.

```
// Pull a batch of messages starting from an offset (topic must be exact, no wildcards)
client.fetch(topic: TopicName, offset: TopicOffset, options?: FetchOptions) → Promise<FetchBatch>

// Pull a batch starting from a timestamp (topic must be exact, no wildcards)
client.fetchFrom(topic: TopicName, timestamp: u64, options?: FetchOptions) → Promise<FetchBatch>

FetchOptions:
  max_count: number             // max messages to return (default: 100)
```

Both methods map directly to dedicated server frames (`Fetch` 0x13 and `FetchFrom` 0x15). Both require only `Fetch` permission. The `has_more` field on `FetchBatch` is computed client-side as `next_offset < head_offset`.

Fetch is **stateless** — no server-side subscriptions, delivery tasks, or offset tracking. The client manages its own offset progression. See [fetch_consumer_guide.md](fetch_consumer_guide.md) for patterns.

### Query (HTTP only)

```
client.getTopicInfo(topic: TopicName) → Promise<TopicInfo>
client.listTopics()                   → Promise<string[]>
```

### Events

Client SDKs should emit or allow registering handlers for:

```
"connect"       → ()                       // authenticated successfully
"disconnect"    → (reason: string)         // connection lost
"error"         → (error: SdkError)        // non-fatal error received
"going_away"    → (reason: GoingAwayReason, deadline_ms: u32, message: string)  // server draining
"reconnecting"  → (attempt: number)        // auto-reconnect in progress
```

### Error type

```
SdkError:
  code:     ErrorCode       // wire error code
  message:  string          // human-readable description
  fatal:    boolean         // whether this error closed the connection
```

---

## Error Handling

### Fatal vs. non-fatal

Client SDKs must distinguish fatal and non-fatal errors:

- **Fatal** (`AuthFailed`, `InvalidFrame`, `FrameTooLarge`, `UnsupportedVersion`, `NotAuthenticated`): the server **will** close the connection after sending the error. The client must transition to `Disconnected` state and may attempt reconnection if configured.
- **Non-fatal** (`PermissionDenied`, `NamespaceNotFound`, `TopicNotFound`, `InternalError`, `ResourceExhausted`, `SlowConsumer`, `OffsetNotAvailable`, `ServerDraining`, `MessageTooLarge`, `InvalidTopic`, `InvalidOperation`): the connection remains open. Surface the error to the caller of the operation that triggered it.

**Special case: `OffsetNotAvailable` (0x000C):** This is an informational warning, not a failure. When a `SubscribeAt` or `SubscribeFrom` requests an offset that has been deleted by retention, the server sends `OffsetNotAvailable` followed by a successful `SubscribeAck`. The subscription is active — the client should log the warning and read `start_offset` from the ack to know where delivery will begin.

**Special case: `ServerDraining` (0x000D):** Returned when the client sends `Subscribe`, `SubscribeAt`, `SubscribeFrom`, or `Reconnect` while the server is draining. This is non-fatal but the connection will close soon. Stop sending subscribe requests, ack in-flight messages, and prepare to reconnect. A client that properly handles `GoingAway` will never see this error.

**Special case: `MessageTooLarge` (0x000E):** Returned when a `Fetch` or `FetchFrom` request hits a message at the requested offset that individually exceeds the frame size budget. The error message includes the stuck offset and message size. SDKs should: (1) surface the error with the offset and sizes, (2) offer a `skip()` convenience that fetches from `offset + 1`, (3) document the HTTP fallback (4MB default budget vs 1MB TCP frame limit), (4) NOT retry the same offset — it will never succeed without a server config change.

**Special case: `InvalidTopic` (0x000F):** Returned when a topic name fails validation — e.g., publishing to a topic containing `*` or `>`, exceeding the 255-byte limit, having empty segments (`..`), or using a wildcard pattern without `CAP_WILDCARD` negotiated. This is non-fatal. SDKs should surface the error and let the caller correct the topic name.

**Special case: `InvalidOperation` (0x0010):** Returned when an operation is not supported in the given context — e.g., `SubscribeAt` with a wildcard pattern (offsets are per-topic), or unsubscribing a specific topic within an active wildcard subscription. This is non-fatal. SDKs should surface the error with a clear message about the unsupported combination.

**Special case: `OffsetNotAvailable` before `FetchResponse`:** When a `Fetch` request specifies an offset below the oldest retained data, the server sends `OffsetNotAvailable` followed by a `FetchResponse` starting from the oldest available offset. This is the same pattern as `SubscribeAt` clamping. SDKs should log the warning and update their stored offset to `FetchResponse.next_offset`.

**GoingAway is NOT an Error:** The `GoingAway` frame (0x10) is a distinct frame type, not an Error frame. It gives the client a grace period to finish work before disconnecting. Do not route `GoingAway` through error handling — handle it as a lifecycle event. See [client_sdk_deploy.md](client_sdk_deploy.md).

### Mapping errors to operations

When the client sends a request and receives an Error frame, the SDK should reject the pending promise/future/callback for that operation. Since the protocol does not include request IDs for correlation, the client must track in-flight operations by type:

- A `Publish` awaiting `PublishAck` — an `Error` frame received instead should reject the publish.
- A `Subscribe` awaiting `SubscribeAck` — an `Error` frame received instead should reject the subscribe.

Implementation note: if multiple operations of the same type can be in-flight simultaneously, the client must match errors to requests by order (FIFO). The server processes frames in order on a single connection.

### Reconnection strategy

Recommended reconnection behavior for TCP clients:

**After receiving GoingAway (planned shutdown):**

1. Ack all in-flight messages, optionally unsubscribe, disconnect gracefully
2. Wait `deadline_ms + uniform_random(0, min(5000, deadline_ms / 2))` — jitter avoids thundering herd
3. Attempt TCP connect + authentication
4. Re-subscribe using `Connect + SubscribeAt` (simple) or `Reconnect` frame (optimized, requires `CAP_RECONNECT`)
5. On failure, apply exponential backoff (base × 2^attempt, capped at 30s)

**After unexpected disconnect (crash / network failure):**

1. Wait `reconnect_interval_ms`
2. Attempt TCP connect + Connect handshake
3. On failure, apply exponential backoff (base × 2^attempt, capped at 30s)
4. On success, re-subscribe to all previously active subscriptions using tracked offsets
5. Emit `"reconnecting"` event on each attempt, `"connect"` on success

**Offset tracking requirement:** Clients MUST track the `offset` field from every `Deliver` frame they acknowledge. This offset is required for resuming consumption via `SubscribeAt` or `Reconnect`. Without it, the client cannot resume from where it left off. See [client_sdk_deploy.md](client_sdk_deploy.md) for details.

---

## Compatibility Testing

This section explains how to run the rust-fafb-q server in your SDK's test pipeline to verify compatibility against a real server.

### Quick start

```bash
# From the rust-fafb-q repo root:
docker compose -f docker-compose.sdk-test.yml up -d

# Wait for healthy:
docker compose -f docker-compose.sdk-test.yml exec fafb-q curl -sf http://localhost:8080/health

# Run your SDK tests against:
#   TCP:  localhost:9876
#   HTTP: localhost:8080

# Tear down:
docker compose -f docker-compose.sdk-test.yml down
```

### Test fixtures

The SDK test server starts with three namespaces, each with a fixed API key. These credentials are stable across versions — SDK tests can hardcode them.

| Namespace | API Key | Permissions | Purpose |
|-----------|---------|-------------|---------|
| `test` | `sdk-test-key-000000000000` | Subscribe, Publish, Query | Full integration testing |
| `publish-only` | `sdk-test-publish-only-00` | Publish | Permission-denied testing (subscribe/query should fail) |
| `query-only` | `sdk-test-query-only-0000` | Query | Permission-denied testing (subscribe/publish should fail) |

### Ports

| Protocol | Default Port | Override env var |
|----------|-------------|------------------|
| TCP | `9876` | `FAFB_TCP_PORT` |
| HTTP | `8080` | `FAFB_HTTP_PORT` |

### Using from an external SDK repo

SDK repos don't need to clone the rust-fafb-q repo. Copy this `docker-compose.yml` into your SDK project — it references a published image (or you can build from source):

**Option A — Build from source (git submodule or clone)**

```yaml
# docker-compose.yml in your SDK repo
services:
  fafb-q:
    build:
      context: ./vendor/rust-fafb-q   # submodule or cloned path
      target: runtime
    ports:
      - "9876:9876"
      - "8080:8080"
    tmpfs:
      - /app/data:size=128m
    environment:
      - FAFB_DATA_DIR=/app/data
      - FAFB_TCP_ADDRESS=0.0.0.0
      - FAFB_HTTP_ADDRESS=0.0.0.0
      - RUST_LOG=warn
    volumes:
      - ./vendor/rust-fafb-q/config/config.sdk-test.toml:/app/config/config.toml:ro
    healthcheck:
      test: ["CMD-SHELL", "curl -sf http://localhost:8080/health || exit 1"]
      interval: 2s
      timeout: 5s
      retries: 10
      start_period: 5s
```

**Option B — Published image (when available)**

```yaml
# docker-compose.yml in your SDK repo
services:
  fafb-q:
    image: ghcr.io/your-org/rust-fafb-q:latest
    ports:
      - "9876:9876"
      - "8080:8080"
    tmpfs:
      - /app/data:size=128m
    environment:
      - FAFB_DATA_DIR=/app/data
      - FAFB_TCP_ADDRESS=0.0.0.0
      - FAFB_HTTP_ADDRESS=0.0.0.0
      - RUST_LOG=warn
    healthcheck:
      test: ["CMD-SHELL", "curl -sf http://localhost:8080/health || exit 1"]
      interval: 2s
      timeout: 5s
      retries: 10
      start_period: 5s
```

### CI pipeline integration

#### GitHub Actions example (Node.js/TypeScript)

```yaml
name: SDK Compat Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      fafb-q:
        image: ghcr.io/your-org/rust-fafb-q:latest
        ports:
          - 9876:9876
          - 8080:8080
        options: >-
          --tmpfs /app/data:size=128m
          --health-cmd "curl -sf http://localhost:8080/health || exit 1"
          --health-interval 2s
          --health-timeout 5s
          --health-retries 10
          --health-start-period 5s
        env:
          FAFB_DATA_DIR: /app/data
          FAFB_TCP_ADDRESS: 0.0.0.0
          FAFB_HTTP_ADDRESS: 0.0.0.0

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npm test
        env:
          FAFB_TCP_HOST: localhost
          FAFB_TCP_PORT: 9876
          FAFB_HTTP_URL: http://localhost:8080
          FAFB_API_KEY: sdk-test-key-000000000000
          FAFB_NAMESPACE: test
```

#### Generic CI pattern (any language)

```bash
#!/usr/bin/env bash
set -euo pipefail

# Start server
docker compose -f docker-compose.sdk-test.yml up -d
trap 'docker compose -f docker-compose.sdk-test.yml down' EXIT

# Wait for health
for i in $(seq 1 30); do
  if curl -sf http://localhost:8080/health > /dev/null 2>&1; then
    echo "Server ready"
    break
  fi
  sleep 1
done

# Run tests (replace with your language's test runner)
export FAFB_TCP_HOST=localhost
export FAFB_TCP_PORT=9876
export FAFB_HTTP_URL=http://localhost:8080
export FAFB_API_KEY=sdk-test-key-000000000000
export FAFB_NAMESPACE=test

npm test          # Node.js
# pytest          # Python
# go test ./...  # Go
# cargo test     # Rust
```

### Recommended environment variables

SDK test suites should read connection details from environment variables to stay portable across local dev and CI. Use these names for consistency across SDKs:

| Variable | Default | Description |
|----------|---------|-------------|
| `FAFB_TCP_HOST` | `localhost` | Server hostname for TCP connections |
| `FAFB_TCP_PORT` | `9876` | TCP port |
| `FAFB_HTTP_URL` | `http://localhost:8080` | HTTP base URL |
| `FAFB_NAMESPACE` | `test` | Namespace to test against |
| `FAFB_API_KEY` | `sdk-test-key-000000000000` | API key for the test namespace |

### Recommended compat test scenarios

At minimum, every SDK should test these scenarios against the real server:

#### TCP

| # | Scenario | Asserts |
|---|----------|---------|
| 1 | Connect with valid credentials | Receives `ConnectAck(success)` |
| 2 | Connect with invalid API key | Receives `ConnectAck(failure)`, connection closed |
| 3 | Connect with wrong namespace | Receives `ConnectAck(failure)`, connection closed |
| 4 | Publish a message | Receives `PublishAck` with valid `MessageId` |
| 5 | Subscribe and receive delivery | Receives `Deliver` after publish to subscribed topic |
| 6 | Ack a delivered message | Server accepts `Ack` without error |
| 7 | Subscribe with consumer group | Receives `SubscribeAck` with group name echoed |
| 8 | Unsubscribe | Receives `UnsubscribeAck`, no further deliveries |
| 9 | Publish without permission | Receives `Error(PermissionDenied)` using `query-only` namespace |
| 10 | Subscribe without permission | Receives `Error(PermissionDenied)` using `publish-only` namespace |
| 11 | Ping/Pong | Send `Ping`, receive `Pong` |
| 12 | Send before auth | Receives `Error(NotAuthenticated)`, connection closed |
| 13 | SubscribeAt from offset | Receives `SubscribeAck` with matching `start_offset`, then `Deliver` from that offset |
| 14 | SubscribeFrom with timestamp | Receives `SubscribeAck` with resolved `start_offset` |
| 15 | SubscribeAt without permission | Receives `Error(PermissionDenied)` |
| 16 | Connect with capabilities | Receives `ConnectAck` with `server_capabilities` reflecting agreed caps |
| 17 | Connect without capabilities (legacy v1) | Receives `ConnectAck` with no capabilities field — still authenticates successfully |
| 18 | Reconnect with valid subscriptions | Receives `ReconnectAck(Success)` with per-subscription results, delivery resumes at correct offsets |
| 19 | Reconnect with partial failure | Receives `ReconnectAck(Partial)` — all subscriptions rolled back, no deliveries |
| 20 | Reconnect with invalid auth | Receives `ReconnectAck(AuthFailed)`, connection closed |
| 21 | Subscribe during drain | Receives `Error(ServerDraining)` — non-fatal |
| 22 | Fetch messages from a topic | Receives `FetchResponse` with correct messages, offsets, and `head_offset` |
| 23 | Fetch with pagination (multiple fetches using `next_offset`) | All messages retrieved across batches, no duplicates, `has_more` transitions from true to false |
| 24 | Fetch empty topic | `FetchResponse` with `count=0`, `next_offset == head_offset` |
| 25 | Fetch without Fetch permission | Receives `Error(PermissionDenied)` |
| 26 | Fetch at clamped offset | Receives `Error(OffsetNotAvailable)` followed by `FetchResponse` starting from oldest available |
| 27 | Fetch with max_count=0 | Receives `Error(InvalidFrame)` |
| 28 | Fetch interleaved with Subscribe on same connection | Both work independently; Fetch does not affect subscription delivery |
| 29 | FetchFrom by timestamp | Receives `FetchResponse` starting from first message at or after the timestamp |
| 30 | FetchFrom with future timestamp | Receives empty `FetchResponse` with `next_offset == head_offset` |
| 31 | Fetch oversized message | Receives `Error(MessageTooLarge)` with offset and size info |
| 32 | Publish with headers → subscribe → deliver | `Deliver` frame includes all published headers when `CAP_HEADERS` negotiated |
| 33 | Publish with headers → fetch | `FetchResponse` messages include headers when `CAP_HEADERS` negotiated |
| 34 | Legacy client publishes → capable client fetches | Fetched messages have empty headers |
| 35 | Publish with reserved `_fafb.` header key | Receives `Error(InvalidFrame)` |
| 36 | Publish with too many headers | Receives `Error(InvalidFrame)` |
| 37 | Nack a delivered message (DLQ enabled) | Receives `NackAck(DeadLettered)`. Message appears in DLQ topic with `_fafb.dlq.*` system headers and original payload. |
| 38 | Nack without `CAP_NACK` | Receives `Error(InvalidFrame)` |
| 39 | Consumer group delivery with headers | `Deliver` frame includes headers for group members |
| 40 | Wildcard subscribe `orders.*` | Receives `SubscribeAck` with `matched_count` > 0, then `Deliver` from all matching topics |
| 41 | Wildcard subscribe auto-subscribe | Subscribe to `orders.*`, then publish to new `orders.apac` → receives delivery from new topic |
| 42 | Wildcard subscribe with consumer group | 2 members subscribed to `orders.*` — per-topic round-robin delivery works |
| 43 | Wildcard `SubscribeAt` rejected | Receives `Error(InvalidOperation)` — offsets not supported with wildcards |
| 44 | Wildcard Fetch rejected | Receives `Error(InvalidTopic)` — wildcards not supported for Fetch |
| 45 | Wildcard without `CAP_WILDCARD` | Receives `Error(InvalidTopic)` when sending wildcard pattern without capability |
| 46 | Unsubscribe wildcard pattern | Delivery stops for all matched topics |
| 47 | Unsubscribe specific topic within wildcard | Receives `Error(InvalidOperation)` |
| 48 | Publish to topic with `*` or `>` | Receives `Error(InvalidTopic)` — wildcard chars not allowed in publish topics |
| 49 | Wildcard `SubscribeFrom` with timestamp | Receives `SubscribeAck` with `matched_count`, per-topic timestamp resolution |

#### HTTP

| # | Scenario | Asserts |
|---|----------|---------|
| 1 | `GET /health/live` | `200 OK` with `{ "status": "alive" }` |
| 1b | `GET /health/ready` | `200 OK` with `{ "status": "ready" }` when server is initialized |
| 2 | Publish via HTTP | `200` with `offset` and `message_id` in JSON |
| 3 | Get topic info | `200` with valid `TopicInfo` JSON |
| 4 | List topics | `200` with namespace and topics array |
| 5 | Missing auth header | `401` with error JSON |
| 6 | Wrong API key | `401` with error JSON |
| 7 | Publish without permission | `403` using `query-only` key |
| 8 | Query without permission | `403` using `publish-only` key |
| 9 | Non-existent namespace | `404` with error JSON |
| 10 | Fetch via HTTP | `200` with `topic`, `next_offset`, `head_offset`, and `messages` array with `payload_base64` |
| 11 | Fetch pagination via HTTP | Chain multiple requests using `next_offset`, verify all messages retrieved |
| 12 | Fetch by timestamp via HTTP | `200` with messages starting from resolved offset |
| 13 | Fetch without Fetch permission | `403` with error JSON |
| 14 | Fetch missing offset parameter | `400` with error JSON |
| 15 | Fetch oversized message via HTTP | `413` with error JSON |
| 16 | HTTP publish with `X-Fafb-*` headers → HTTP fetch | Fetch response messages include `headers` array with correct keys/values |
| 17 | HTTP publish with reserved `X-Fafb-_fafb.*` | `400` with error JSON |
| 18 | HTTP publish with too many `X-Fafb-*` headers | `400` with error JSON |
| 19 | HTTP fetch binary header values | `value_base64` field present instead of `value` |
| 20 | HTTP fetch duplicate header keys | All duplicate keys preserved in `headers` array |

---

## Implementation Checklist

Use this checklist when building a new client SDK. Each item represents a requirement that should be tested.

### Codec

- [ ] Encode/decode frame envelope (4-byte length + 1-byte type + payload)
- [ ] Encode/decode length-prefixed strings (2-byte `u16` BE + UTF-8)
- [ ] Encode/decode length-prefixed payloads (4-byte `u32` BE + raw bytes)
- [ ] Encode/decode all 23 message types with correct field order (including GoingAway, Reconnect, ReconnectAck, Fetch, FetchResponse, FetchFrom, Nack, NackAck)
- [ ] Handle partial frame reads (buffering until complete frame received)
- [ ] Reject frames exceeding max frame size before reading payload
- [ ] Reject unknown message type bytes
- [ ] All multi-byte integers are big-endian

### State machine

- [ ] Enforce valid message sends per connection state
- [ ] Transition on `ConnectAck(success)` → `Authenticated`
- [ ] Transition on `ConnectAck(failure)` → `Disconnected`
- [ ] Transition on `ReconnectAck(Success)` → `Subscribed` (or `Authenticated` if no subscriptions)
- [ ] Transition on `ReconnectAck(Partial)` → `Authenticated` (no active subscriptions)
- [ ] Transition on `ReconnectAck(AuthFailed)` → `Disconnected`
- [ ] Transition on `GoingAway` received → `Draining`
- [ ] Track active subscriptions, transition between `Authenticated` / `Subscribed`
- [ ] In `Draining` state: reject Subscribe/SubscribeAt/SubscribeFrom, allow Publish/Ack/Unsubscribe/Ping
- [ ] Handle fatal errors: transition to `Disconnected`, no further sends

### Operations

- [ ] `connect()` sends `Connect` frame and awaits `ConnectAck`
- [ ] `subscribe()` sends `Subscribe` and awaits `SubscribeAck` (check `start_offset`)
- [ ] `subscribeAt()` sends `SubscribeAt` and awaits `SubscribeAck` (check `start_offset` — may be clamped)
- [ ] `subscribeFrom()` sends `SubscribeFrom` and awaits `SubscribeAck` (check `start_offset`)
- [ ] Handle non-fatal `OffsetNotAvailable` error before `SubscribeAck` (offset was clamped)
- [ ] `unsubscribe()` sends `Unsubscribe` and awaits `UnsubscribeAck`
- [ ] `publish()` sends `Publish` and awaits `PublishAck`
- [ ] `ack()` sends `Ack` (fire-and-forget, no server response)
- [ ] Ping/Pong: respond to server `Ping` with `Pong`
- [ ] Keepalive: send periodic `Ping` at `ping_interval_ms` when connection is idle (MANDATORY for long-lived connections)

### Fetch (pull consumption)

- [ ] Encode/decode Fetch frame (0x13): topic + offset + max_count
- [ ] Encode/decode FetchFrom frame (0x15): topic + timestamp_millis + max_count
- [ ] Encode/decode FetchResponse frame (0x14): topic + next_offset + head_offset + count + repeated messages
- [ ] Compute `has_more` from `next_offset < head_offset`
- [ ] Handle fewer-than-requested messages (frame size truncation)
- [ ] Handle `OffsetNotAvailable` warning before FetchResponse (offset clamping)
- [ ] Handle `MessageTooLarge` error — surface offset and sizes, offer skip
- [ ] Handle empty FetchResponse (count=0) — topic empty or caught up
- [ ] Expose `fetch(topic, offset, options)` in public API
- [ ] Expose `fetchFrom(topic, timestamp, options)` in public API
- [ ] Track next_offset from response for pagination

### Message headers

- [ ] Encode/decode header section: `u16` header count + repeated `(u16 key_len, key, u32 value_len, value)` entries
- [ ] Conditionally include header section in Publish, Deliver, FetchResponse when `CAP_HEADERS` negotiated
- [ ] Omit header section entirely when `CAP_HEADERS` not negotiated (legacy format)
- [ ] Validate header limits before sending Publish: count ≤ 64, key ≤ 256B, value ≤ 8KB, total ≤ 32KB
- [ ] Reject reserved `_fafb.` prefix in client-submitted header keys (client-side validation)
- [ ] Expose `headers` field on `DeliveredMessage` and `FetchedMessage`
- [ ] Expose `headers` parameter on `publish()` method
- [ ] HTTP publish: send headers as `X-Fafb-{key}` HTTP headers
- [ ] HTTP fetch: parse `headers` array from JSON response, handle `value` vs `value_base64` fields

### Nack (explicit message rejection)

- [ ] Encode/decode Nack frame (0x16): message_id + reason string
- [ ] Encode/decode NackAck frame (0x17): message_id + status byte
- [ ] Require `CAP_NACK` before sending Nack — raise client-side error if not negotiated
- [ ] Expose `nack(reason?)` method on `DeliveredMessage`
- [ ] Handle NackAck response and surface `NackAckStatus` to caller
- [ ] Truncate reason to 1024 bytes before sending

### HTTP fetch (if supported)

- [ ] HTTP fetch: parse base64-encoded payloads in `payload_base64` field
- [ ] HTTP fetch: handle pagination with `next_offset`
- [ ] HTTP fetch: support both `offset` and `timestamp` query params
- [ ] HTTP fetch: validate `max_count` ≤ 10,000 (HTTP-specific limit)
- [ ] HTTP fetch: handle `warning` field for offset clamping (TCP uses separate error frame)

### Wildcard subscriptions

- [ ] Detect wildcard patterns in topic (`*` or `>` segments) — require `CAP_WILDCARD` negotiated
- [ ] Reject `subscribeAt()` with wildcard pattern client-side (raise error before sending)
- [ ] Reject `fetch()` / `fetchFrom()` with wildcard pattern client-side
- [ ] Parse `matched_count` from `SubscribeAck` when `CAP_WILDCARD` negotiated (trailing `u16` after group)
- [ ] Expose `matched_count` on `Subscription` object
- [ ] Handle `ErrorCode::InvalidTopic` (0x000F) — non-fatal, topic validation failure
- [ ] Handle `ErrorCode::InvalidOperation` (0x0010) — non-fatal, unsupported operation context
- [ ] Unsubscribe with original pattern (not expanded topic names)
- [ ] Document wildcard Reconnect limitation: re-subscribes from head, not per-topic offsets

### Error handling

- [ ] Parse `Error` frames and construct typed `SdkError`
- [ ] Distinguish fatal vs. non-fatal by error code
- [ ] Reject pending operations on error
- [ ] Surface non-fatal errors without closing connection

### HTTP (if supported)

- [ ] `Authorization: Bearer <api_key>` header on all requests (except `/health`)
- [ ] Parse JSON responses into typed objects
- [ ] Parse JSON error responses into `SdkError`
- [ ] Map HTTP status codes to error codes

### Deploy support

- [ ] Send `capabilities` bitmask in `Connect` frame (`CAP_GOING_AWAY | CAP_RECONNECT | CAP_NACK | CAP_HEADERS | CAP_WILDCARD`)
- [ ] Store `server_capabilities` from `ConnectAck` for session lifetime
- [ ] Handle `GoingAway` frame: emit event, ack in-flight, disconnect gracefully
- [ ] Handle `ServerDraining` error: stop subscribing, prepare to reconnect
- [ ] Track `offset` from every acked `Deliver` frame per topic per group
- [ ] Implement `Reconnect` frame (optional optimization for high subscription count)
- [ ] Handle `ReconnectAck`: Success (delivery starts), Partial (rollback, fall back to Connect+SubscribeAt), AuthFailed (connection closed)
- [ ] Check per-subscription `OffsetClamped` warnings in ReconnectAck results

### Resilience

- [ ] Auto-reconnect with exponential backoff
- [ ] GoingAway-aware reconnection: use deadline_ms + jitter to avoid thundering herd
- [ ] Re-subscribe after reconnection using tracked offsets (`SubscribeAt` or `Reconnect`)
- [ ] Keepalive via Ping/Pong with configurable interval and timeout
- [ ] Detect dead connection (no Pong within timeout)

---

## Version History

| Version | Date | Description |
|---------|------|-------------|
| 1.0 | 2026-03-20 | Initial client SDK specification for protocol v1 |
| 1.1 | 2026-03-22 | Graceful deployment support: capability negotiation (Connect/ConnectAck), GoingAway frame (0x10), Reconnect/ReconnectAck frames (0x11/0x12), ServerDraining error code (0x000D), Draining connection state, offset tracking requirement, reconnect backoff with jitter. See [client_sdk_deploy.md](client_sdk_deploy.md) for implementation guide. |
| 1.2 | 2026-03-23 | Pull-based fetch protocol: Fetch (0x13), FetchFrom (0x15), FetchResponse (0x14) frames, Permission::Fetch, ErrorCode::MessageTooLarge (0x000E), HTTP `GET /v1/{ns}/{topic}/fetch` endpoint with base64 payloads. See [fetch_consumer_guide.md](fetch_consumer_guide.md) for SDK implementation patterns. |
| 1.3 | 2026-03-25 | Dead letter queue: Nack (0x16) and NackAck (0x17) frames for explicit message rejection, `CAP_NACK` capability (bit 2), `NackAckStatus` enum. Message headers: `CAP_HEADERS` capability (bit 3), header sections on Publish/Deliver/FetchResponse frames, `Header` type, header validation limits, `_fafb.*` reserved prefix, HTTP `X-Fafb-*` header passthrough, JSON `headers` array in fetch responses, DLQ system headers (`_fafb.dlq.*`) replacing binary envelope format. |
| 1.4 | 2026-03-25 | Wildcard subscriptions: `CAP_WILDCARD` capability (bit 4), NATS-style `*` (one segment) and `>` (trailing segments) patterns in Subscribe/SubscribeFrom, `SubscribeAck.matched_count` field, auto-subscribe on new topic creation, wildcard consumer groups with per-topic round-robin, `ErrorCode::InvalidTopic` (0x000F) and `ErrorCode::InvalidOperation` (0x0010), topic name validation (255 byte max, no empty segments, no `*`/`>` on publish), `max_topics_per_wildcard` config limit. See [wildcard_subscriptions.md](wildcard_subscriptions.md) for usage guide. |
