# Synapse: The Client Layer

> Schema-driven CLI for Plexus RPC servers. The entire CLI writes itself at runtime from backend JSON schemas.

## Overview

Synapse (`synapse/`) is written in Haskell (GHC2021, cabal). It sits above the Plexus RPC protocol and provides progressive discovery, IR-driven parameter parsing, and template-based output rendering. No code generation step, no static configuration.

```
Plexus RPC Servers (Substrate, etc.)
           ↑
    Plexus Protocol (WebSocket JSON-RPC 2.0)
           ↑
    Synapse CLI
           ↑
    User shell / TTY / Agent
```

## Request Lifecycle

```
CLI invocation
    ↓
1. Parse flags (-H, -P, --json, etc.) + collect path segments + --key value params
    ↓
2. Backend discovery: ping host:port, call _info RPC to get backend name
    ↓
3. Schema navigation: recursive descent through plugin tree, cache by content hash
    ↓
4. Parameter transformation: path expansion, env var expansion, smart defaults
    ↓
5. IR construction: walk schema tree, extract types, deduplicate across namespaces
    ↓
6. IR-driven parameter parsing: flat --key value pairs → nested typed JSON
    ↓
7. Method invocation: JSON-RPC 2.0 subscription over WebSocket
    ↓
8. Streaming response: each PlexusStreamItem rendered via Mustache template or JSON
```

## Progressive Discovery (Navigation)

Synapse navigates the plugin tree via recursive descent. Schemas are fetched lazily and cached by content hash:

```bash
synapse -P 4444                              # List available backends (registry)
synapse -P 4444 substrate                     # Show root schema (plugins)
synapse -P 4444 substrate echo                # Show echo plugin's methods
synapse -P 4444 substrate echo once --message "Hello"  # Invoke method
```

Navigation uses a **paramorphism** (fold with context) over `SchemaF`, the base functor:
- `PluginF` — interior nodes with children (hub plugins)
- `MethodF` — leaves (invocable methods)

**Cycle detection**: Tracks visited hashes in a `HashSet`; throws `NavError::Cycle` if re-encountered. **Caching**: `IORef (HashMap PluginHash PluginSchema)` avoids re-fetching.

**Auto-help**: Methods with required params but no user-supplied params show help instead of invoking. Methods with no required params auto-invoke.

**Path normalization**: Hyphens in path segments and param keys are normalized to underscores. Leading namespace is stripped if it matches root (so `synapse plexus cone` = `synapse cone`).

## The IR System

The Intermediate Representation is a **deduplicated, global type registry** built from the schema tree. It powers parameter parsing, help text, template generation, and code generation.

```haskell
data IR = IR
  { irVersion      :: Text                      -- "2.0"
  , irBackend      :: Text                      -- "substrate"
  , irHash         :: Maybe Text                -- Root plugin hash
  , irMetadata     :: Maybe GenerationMetadata  -- Generator info + timestamp
  , irTypes        :: Map Text TypeDef          -- All types, keyed by "ns.TypeName"
  , irMethods      :: Map Text MethodDef        -- All methods, keyed by "ns.method"
  , irPlugins      :: Map Text [Text]           -- Plugin → method names
  , irPluginHashes :: Maybe (Map Text PluginHashInfo)
  }
```

### Type Kinds

| Kind | Description |
|------|-------------|
| `KindStruct` | Fields with types, descriptions, required flags |
| `KindEnum` | Discriminated unions (tagged via discriminator field) |
| `KindStringEnum` | Simple string enums |
| `KindAlias` | Type aliases |
| `KindPrimitive` | string, integer, boolean, etc. with optional format |

### Type References

| Ref | Description |
|-----|-------------|
| `RefNamed` | Reference to a named type (deduplicated) |
| `RefPrimitive` | "string", "integer", etc. with optional format |
| `RefArray` | Arrays |
| `RefOptional` | Nullable |
| `RefAny` | `serde_json::Value` (intentionally dynamic) |
| `RefUnknown` | Schema gap (should warn) |

### Deduplication

Multiple namespaces may define identical types. Synapse deduplicates by:

1. **Hash by structure**: Two `TypeDef`s are duplicates if they have the same name and kind structure
2. **Pick canonical**: Prefer parent namespace (the "owner"), else prefer shortest namespace
3. **Update references**: All `RefNamed` pointers updated to canonical qualified name

### Streaming Inference

A method is streaming if its return type is a discriminated union with >1 non-error variant. Recorded in `mdStreaming :: Bool` on the `MethodDef`.

### Method Definition

```haskell
data MethodDef = MethodDef
  { mdName        :: Text          -- "chat"
  , mdFullPath    :: Text          -- "cone.chat"
  , mdNamespace   :: Text          -- "cone"
  , mdDescription :: Maybe Text
  , mdStreaming   :: Bool          -- Inferred from return type
  , mdParams      :: [ParamDef]    -- Input parameters
  , mdReturns     :: TypeRef       -- Return type
  , mdBidirType   :: Maybe TypeRef -- Bidirectional request type
  , mdBidirResponseType :: Maybe TypeRef
  , mdBidirResponseSchema :: Maybe Value
  }
```

## IR-Driven Parameter Parsing

Flat CLI flags are transformed into nested JSON using type information from the IR:

```bash
# Dotted keys for nested params:
--identifier.type by_name --identifier.name foo --prompt "hello"
# → {"identifier": {"type": "by_name", "name": "foo"}, "prompt": "hello"}

# Repeated flags for arrays:
--tags backend --tags critical
# → {"tags": ["backend", "critical"]}

# Boolean flags (bare):
--force
# → {"force": true}
```

### Parsing Algorithm

1. **Group by prefix**: Split dotted keys, group by first segment
2. **Look up TypeRef**: For each param, find its type in `MethodDef.mdParams`
3. **Build typed value**: Recursively process sub-keys using type information
   - **Struct**: Build object from sub-keys
   - **Enum**: Discriminator field selects variant; only that variant's fields accepted
   - **Array**: Collect repeated flags
   - **Optional**: Field can be omitted
   - **Primitive**: Type-infer from string (`"true"` → bool, numeric → number)

**Error reporting** includes edit-distance suggestions for typos.

## Parameter Transforms

Before IR parsing, optional transforms run on raw params:

1. **Path expansion**: `.` → CWD, `~/` → home. Triggered by param names: `path`, `working_dir`, `output_dir`, `file_path`, `dir`, `directory`, `workdir`
2. **Env var expansion**: `$VAR` replaced with environment value
3. **Smart defaults**: Missing required path params get CWD injected
4. **Boolean defaults**: Bare `--flag` becomes `"true"` if param is boolean type

Transforms are fail-safe: on any error, the original value passes through.

## Template Rendering

Streaming responses are rendered via Mustache templates.

### Resolution Order

1. `.substrate/templates/{namespace}/{method}.mustache` (project-local)
2. `~/.config/synapse/templates/{namespace}/{method}.mustache` (user)
3. Fallback: YAML-like pretty printing

### Output Modes

| Flag | Mode | Behavior |
|------|------|----------|
| (none) | Template | Mustache template rendering |
| `--json` / `-j` | Raw JSON | Full `PlexusStreamItem` JSON per line |
| `--raw` | Content only | Just the content JSON, skip wrapper metadata |

Templates are auto-generated from the IR via `synapse _self template generate`. For enum returns, all variants appear as Mustache sections in one unified template.

## Backend Discovery

Synapse discovers backends through the **registry**:

```bash
synapse _self scan          # Scan ports 4440-4459 for backends
synapse                     # List all registered backends (via registry.list RPC)
```

Discovery flow:
1. Connect to registry at host:port (default `127.0.0.1:4444`)
2. Call `registry.list` to get all backends
3. Ping each in parallel (300ms timeout) to check reachability
4. Display with status

## Bidirectional Communication

For methods that send requests to the client mid-execution (confirmations, prompts):

**Modes** (auto-detected or configured):

| Mode | Trigger | Behavior |
|------|---------|----------|
| `BidirInteractive` | stdin is TTY | Prompt user via terminal |
| `BidirJson` | stdin is pipe | Output request JSON, read response from stdin |
| `BidirAutoCancel` | non-interactive fallback | Automatically cancel |
| `BidirDefaults` | configured | Use default values from request |
| `BidirCmd <cmd>` | configured | Spawn subprocess per request |
| `BidirRespond` | configured | Print request, agent responds via `synapse <backend> respond` |

When a `StreamRequest` arrives:
1. Call the bidir handler with request ID and data
2. If handler returns `Just response`, send via `{backend}.respond` RPC
3. If `Nothing`, caller handles (for `BidirRespond` mode)

## Effect Stack

```haskell
type SynapseM a = ExceptT SynapseError (ReaderT SynapseEnv IO) a

data SynapseEnv = SynapseEnv
  { seHost    :: Text
  , sePort    :: Int
  , seBackend :: Text
  , seCache   :: IORef (HashMap PluginHash PluginSchema)
  , seVisited :: IORef (HashSet PluginHash)
  }
```

### Error Types

| Error | Description |
|-------|-------------|
| `NavError` | NotFound, MethodNotTerminal, Cycle, FetchError |
| `TransportError` | Generic transport error |
| `TransportErrorContext` | With connection details (host, port, category) |
| `ParseError` | Parameter parsing error |
| `ValidationError` | Validation error |
| `BackendError` | Backend discovery error |

Errors are rendered with context-aware suggestions: navigation errors show available commands, parameter errors show similar names, transport errors show troubleshooting steps.

## Meta-Commands

```bash
synapse _self scan                        # Scan ports 4440-4459
synapse _self template list [pattern]     # List templates
synapse _self template show <ns.method>   # Show template content
synapse _self template generate [pattern] # Generate templates from IR
synapse _self template delete <pattern>   # Delete templates
synapse _self template reload             # Clear template cache (stub)
```

## CLI Options

```bash
synapse [OPTIONS] <backend> <path...> [--param value ...]
```

Options must come BEFORE the backend (`noIntersperse`):

| Option | Description |
|--------|-------------|
| `-H HOST` | Registry host (default: `127.0.0.1`) |
| `-P PORT` | Registry port (default: `4444`) |
| `-j` / `--json` | Raw JSON stream output |
| `--raw` | Content JSON only |
| `-n` / `--dry-run` | Show JSON-RPC request without sending |
| `-s` / `--schema` | Fetch raw schema JSON |
| `-g` / `--generate-templates` | Generate Mustache templates from IR |
| `-i` / `--emit-ir` | Emit IR JSON for code generation |
| `-p JSON` / `--params JSON` | Parameters as raw JSON object |
| `-r JSON` / `--rpc JSON` | Raw JSON-RPC passthrough |

## Source Layout

```
synapse/
  app/Main.hs                    — CLI entry, dispatch, output rendering
  src/Synapse/
    Monad.hs                     — SynapseM effect stack, environment, errors
    Transport.hs                 — WebSocket JSON-RPC bridge (via plexus-protocol)
    Renderer.hs                  — Output rendering (template resolution, pretty-print)
    Schema/{Types,Functor}.hs    — Core types, SchemaF base functor
    Algebra/{Recursion,Navigate,Walk}.hs — Recursion schemes, navigation
    IR/{Types,Builder}.hs        — Intermediate representation, type deduplication
    CLI/{Help,Parse,Transform,Template}.hs — CLI layer
    Backend/Discovery.hs         — Registry-based backend discovery
    Bidir.hs                     — Bidirectional request handling
    Self/{Commands,Template,Pattern}.hs — _self meta-commands
```

## Test Suites

| Suite | Scope |
|-------|-------|
| cli-test | Integration (requires backend at localhost:4444) |
| ir-test | IR builder, type resolution, help text |
| parse-test | CLI parameter parsing |
| path-normalization-test | Hyphen/underscore normalization |
| typeref-json | TypeRef JSON serialization |
| bidir-test | Bidirectional communication (unit) |
| bidir-integration-test | Bidirectional communication (integration) |
