# IR-Based CLI Implementation Summary

**Status**: Completed
**Date**: 2026-01-06
**Branch**: `feature/cli-from-ir` (synapse)
**Linked**: substrate, synapse

## Overview

This document summarizes the complete work done in the `feature/cli-from-ir` branch of synapse. This transforms the CLI from ad-hoc JSON Schema parsing to a unified IR-driven approach, where all CLI operations (help rendering, parameter parsing, support checking) consume the same intermediate representation built from the schema.

## Problem Statement

Before this work, synapse had three independent JSON Schema consumers:

```
                         JSON Schema (from plexus.schema)
                                    |
              +---------------------+---------------------+
              |                     |                     |
              v                     v                     v
      CLI Help Rendering      Param Parsing         Support Check
   (ad-hoc extractTypeDesc)  (flat key-value)    (pattern matching)
              |                     |                     |
              v                     v                     v
   Shows: --identifier <any>  Produces flat JSON   Rejects all oneOf
```

### Concrete Failure Example

```bash
synapse substrate cone chat --identifier.type by_name --identifier.name haiku35 --prompt "hi"
# Produced: {"identifier.type": "by_name", "identifier.name": "haiku35", "prompt": "hi"}
# Expected: {"identifier": {"type": "by_name", "name": "haiku35"}, "prompt": "hi"}
```

The CLI accepted dotted flags but produced flat JSON with literal dotted keys, rather than nested objects. The server rejected this with an internal error.

### Problems Summary

| Issue | Impact |
|-------|--------|
| **Duplicated parsing logic** | Same JSON Schema patterns detected in 3 places |
| **Help/Parse decoupled** | Help showed `--identifier.type by_name` but parser didn't understand nesting |
| **Tagged unions unsupported** | `oneOf` schemas rejected entirely |
| **No type-aware parsing** | All values treated as strings |

## Solution: Unified IR Layer

All CLI operations now use the same IR:

```
JSON Schema (wire) --> IR Builder --> IR (MethodDef, TypeRef, TypeDef)
                                              |
                        +---------------------+---------------------+
                        |                     |                     |
                        v                     v                     v
                   CLI.Help              CLI.Parse             CLI.Support
                   (renders)             (parses)              (checks)
```

The JSON Schema to IR transformation happens exactly once. All downstream consumers share the same structured representation.

## Modules Created

### 1. `Synapse.CLI.Help` (264 lines)

Renders method help from IR with tagged union expansion.

**Key functions:**
- `renderMethodHelp :: IR -> MethodDef -> Text` - Full method documentation
- `renderParamHelp :: IR -> ParamDef -> [Text]` - Parameter with type expansion
- `expandType :: IR -> Text -> TypeRef -> [Text]` - Expands `RefNamed` to show variants
- `renderTypeRef :: IR -> TypeRef -> Text` - Renders type signatures

**Configuration:**
```haskell
data HelpStyle = HelpStyle
  { hsIndent      :: Int    -- Spaces per indent level
  , hsParamWidth  :: Int    -- Width for parameter column
  , hsShowFormats :: Bool   -- Show format hints (uuid, int64, etc.)
  , hsExpandEnums :: Bool   -- Expand enum variants inline
  }
```

**Example output:**
```
cone.chat [streaming]

Chat with a cone - appends prompt to context, calls LLM, advances head

Parameters:

  --identifier <ConeIdentifier>  (required)
      Cone name or UUID (e.g., 'my-assistant' or '550e8400-e29b-...')
      Either:
        --identifier.type by_name --identifier.name <string>
        --identifier.type by_id --identifier.id <uuid>

  --prompt <string>  (required)
      User message / prompt to send to the LLM

  --ephemeral <boolean?>  (optional)
      If true, creates nodes but doesn't advance head and marks for deletion
```

### 2. `Synapse.CLI.Parse` (315 lines)

Parses flat `--key value` pairs into properly nested JSON using IR structure.

**Key functions:**
- `parseParams :: IR -> MethodDef -> [(Text, Text)] -> Either [ParseError] Value`
- `groupByPrefix :: [(Text, Text)] -> Map Text [(Text, Text)]` - Splits dotted keys
- `buildFromTypeDef :: IR -> Text -> TypeDef -> [(Text, Text)] -> Either ParseError Value`
- `buildVariant :: IR -> Text -> Text -> VariantDef -> [(Text, Text)] -> Either ParseError Value`

**Error types:**
```haskell
data ParseError
  = UnknownParam Text              -- Flag doesn't match any known parameter
  | MissingRequired Text           -- Required parameter was not provided
  | InvalidValue Text Text         -- Invalid value for parameter (param, reason)
  | AmbiguousVariant Text [Text]   -- Multiple variants could match
  | MissingDiscriminator Text Text -- Enum type missing discriminator value
  | UnknownVariant Text Text [Text]-- Unknown variant name
  | TypeNotFound Text              -- Referenced type not found in IR
```

**Algorithm:**
1. Group flags by prefix: `identifier.type` -> `("identifier", [("type", v)])`
2. Look up param in IR: `identifier` -> `RefNamed "ConeIdentifier"`
3. Look up type: `ConeIdentifier` -> `KindEnum "type" [variants]`
4. Find variant by discriminator value
5. Build nested object with discriminator + variant fields
6. Apply type coercion for primitives (string/boolean/number)

**Example transformation:**
```
Input:  [("identifier.type", "by_name"), ("identifier.name", "haiku"), ("prompt", "hi")]
Output: {"identifier": {"type": "by_name", "name": "haiku"}, "prompt": "hi"}
```

### 3. `Synapse.CLI.Support` (270 lines)

Checks CLI representability from IR types.

**Key functions:**
- `methodSupport :: IR -> MethodDef -> SupportLevel`
- `canCLIRepresent :: IR -> TypeRef -> Bool`
- `canCLIRepresentVariant :: IR -> VariantDef -> Bool`
- `canCLIRepresentField :: IR -> FieldDef -> Either SupportReason ()`

**Support levels:**
```haskell
data SupportLevel
  = FullSupport              -- All params can be CLI flags
  | PartialSupport [Text]    -- Some optional params need JSON
  | NoSupport [Text]         -- Required params need JSON
```

**Support reasons:**
```haskell
data SupportReason
  = ReasonArray          -- Arrays need JSON or special handling
  | ReasonNested         -- Deeply nested structure
  | ReasonAny            -- Type is dynamic (any)
  | ReasonUnknown        -- Type information missing
  | ReasonComplexUnion   -- Union with non-representable variants
```

**Representability rules:**
- Primitives: Always representable (string, integer, number, boolean)
- UUID/formatted strings: Representable as strings
- Optional T: Representable if T is representable (flag is omittable)
- Arrays: NOT representable as simple flags (need JSON or repeated flags)
- Structs: Representable if all required fields are representable
- Enums (discriminated): Representable via `--field.type variant --field.x ...`
- String enums: Representable (finite set of string values)
- Any/Unknown: NOT representable (need JSON input)

## Integration Tests

`test/IRSpec.hs` validates against live plexus backend (200 lines):

```
Schema fetching
  [check] builds IR from root
  [check] IR contains methods
  [check] IR contains types
  [check] IR contains plugins
Method coverage
  [check] all methods have help text
  [check] all type refs resolve
  [check] all methods have support level
Specific methods
  [check] cone.chat has identifier param
  [check] cone.chat expands ConeIdentifier
  [check] echo.once has simple params
  [check] echo.once has message param
Type resolution
  [check] no dangling RefNamed in params
  [check] reports unresolved return type refs (informational)
```

**Configuration:** `PLEXUS_PORT` env var (default 4444)

**Key invariants verified:**
1. IR builds successfully from schema
2. All methods produce non-empty help text
3. All `RefNamed` in params resolve to types in `irTypes`
4. All methods categorize to valid `SupportLevel`
5. Critical paths like `ConeIdentifier` expand correctly

## Commits

1. **`feat: IR-based CLI help with tagged union expansion`** (`5b2c07b`)
   - Added `Synapse.CLI.Help` module
   - Added `Synapse.CLI.Support` module
   - IR-driven help rendering with variant expansion

2. **`feat: IR-driven parameter parsing for nested JSON from flat flags`** (`1afad6e`)
   - Added `Synapse.CLI.Parse` module
   - Proper nesting of dotted flags into JSON objects
   - Type-aware value coercion

## Example Usage

**Before (broken):**
```bash
synapse substrate cone chat --identifier.type by_name --identifier.name haiku --prompt "hi"
# Error: Internal error (flat JSON sent to server)
```

**After (working):**
```bash
synapse substrate cone chat --identifier.type by_name --identifier.name haiku --prompt "hi"
# Success: {"identifier": {"type": "by_name", "name": "haiku"}, "prompt": "hi"}
# Server responds with LLM output
```

**Help output (before):**
```
cone.chat - Chat with a cone

  --identifier <any>     Cone name or UUID
  --prompt <string>      User message
```

**Help output (after):**
```
cone.chat [streaming]

Chat with a cone - appends prompt to context, calls LLM, advances head

Parameters:

  --identifier <ConeIdentifier>  (required)
      Cone name or UUID (e.g., 'my-assistant' or '550e8400-e29b-...')
      Either:
        --identifier.type by_name --identifier.name <string>
        --identifier.type by_id --identifier.id <uuid>

  --prompt <string>  (required)
      User message / prompt to send to the LLM
```

## Discoveries

### Schema Union Asymmetries

Testing revealed `health.schema` returns `SchemaResult` using `anyOf` (untagged union) rather than `oneOf` (tagged union). The IR Builder only handled `oneOf`.

**Key insight:** For `anyOf` with `$ref` variants, the **type name is the discriminator**:
```json
{"anyOf": [{"$ref": "#/$defs/PluginSchema"}, {"$ref": "#/$defs/MethodSchema"}]}
```
Discriminator is implicitly "PluginSchema" vs "MethodSchema".

See: `schema-union-asymmetries.md` for full analysis.

### RefAny vs RefUnknown Distinction

The IR Builder now distinguishes between:
- `RefAny`: Schema present but no type constraints (intentionally dynamic, e.g., `serde_json::Value`)
- `RefUnknown`: No schema at all (schema gap, should warn)

This distinction matters for support checking - `RefAny` is a deliberate design choice, while `RefUnknown` indicates missing schema information.

## Architecture Relationship

This work validates and implements concepts from:

| Document | Concept | How This Work Relates |
|----------|---------|----------------------|
| `unified-cli-tree.md` | ParamF concept | IR + CLI modules are the realization |
| `structured-params.md` | Client-side structured types | IR is the structured representation |
| `ir-codegen-chain.md` | IR powers codegen | Same IR now powers CLI too |
| `ir-based-cli.md` | Proposal for IR-based CLI | This is the implementation |

The CLI modules consume the same IR that codegen uses:

```
JSON Schema
    |
    v
IR Builder
    |
    v
    IR --------------------------+
    |                            |
    +-- hub-codegen (TypeScript) |
    +-- future codegen           |
    |                            |
    +-- CLI features <-----------+
         +-- Help rendering (CLI.Help)
         +-- Param parsing (CLI.Parse)
         +-- Support checking (CLI.Support)
```

## Future Work

1. **Tab completion** - Use IR to complete enum values, variant names
2. **`anyOf` handling** - Add untagged union support to IR Builder
3. **Array syntax** - `--items[0] foo --items[1] bar`
4. **Shorthand** - `--identifier haiku` as sugar for `--identifier.type by_name --identifier.name haiku`
5. **Validation messages** - Use IR type info for better error messages

## Summary Table

| Aspect | Before | After |
|--------|--------|-------|
| Schema parsing | 3 independent parsers | Single IR Builder |
| Param output | Flat JSON `{"a.b": "c"}` | Nested JSON `{"a": {"b": "c"}}` |
| Help/Parse coupling | Decoupled, inconsistent | Tightly coupled via IR |
| Tagged unions | Unsupported (all `oneOf` rejected) | Full expansion with variants |
| Type coercion | All values as strings | Type-aware (bool, number, etc.) |
| Integration tests | None | 13 tests against live backend |
| String enums | Rejected | Fully supported |
| Support checking | Re-parses schema | Uses IR |

## File Locations

- `/Users/user/dev/controlflow/hypermemetic/synapse-cli-from-ir/src/Synapse/CLI/Help.hs`
- `/Users/user/dev/controlflow/hypermemetic/synapse-cli-from-ir/src/Synapse/CLI/Parse.hs`
- `/Users/user/dev/controlflow/hypermemetic/synapse-cli-from-ir/src/Synapse/CLI/Support.hs`
- `/Users/user/dev/controlflow/hypermemetic/synapse-cli-from-ir/src/Synapse/IR/Types.hs`
- `/Users/user/dev/controlflow/hypermemetic/synapse-cli-from-ir/src/Synapse/IR/Builder.hs`
- `/Users/user/dev/controlflow/hypermemetic/synapse-cli-from-ir/test/IRSpec.hs`
