# Nexus Integration — Output & Embedding Design

This document describes how external systems consume Nexus's network descriptions. The interpreter produces an internal representation; integration layers expose it to consumers in various forms.

## Architecture

```
                    ┌─────────────────────────────────┐
                    │         Nexus Interpreter        │
                    │        (produces Network)        │
                    └──────────────┬──────────────────┘
                                   │
                    ┌──────────────▼──────────────────┐
                    │      Public Rust API (crate)     │
                    │   Network, Node, Connection,     │
                    │   Port, PortRef, Warning         │
                    └──┬──────────┬──────────┬────────┘
                       │          │          │
              ┌────────▼──┐ ┌────▼─────┐ ┌──▼──────────┐
              │  C ABI FFI │ │   JSON   │ │  DOT/other  │
              │  (libffi)  │ │  (serde) │ │  (formatters)│
              └────────┬──┘ └────┬─────┘ └──┬──────────┘
                       │         │          │
              C, C++,  │   Any   │   Graph  │
              Python,  │  lang   │   viz    │
              Go, ...  │         │   tools  │
```

The Rust API is the foundation. All other integration paths are thin adapters on top of it.

## Layer 1: Public Rust API

The primary interface. The interpreter evaluates a Nexus program and returns a `Network` — a self-contained, serializable representation of the final network.

### Data model

Nodes and groups are unified into a single `Node` type. A group is a node with non-empty `children`; a leaf node has `children: []`. Both have a name, a type, and ports — the same fields serve both roles.

`NodeId` is a `u64` assigned during interpretation. It serves as a stable, O(1) machine-friendly identifier for connections. Human-readable identity comes from the name and tree position; `NodeId` is for programmatic lookups without tree traversal.

Connections are owned by the `Network`, not by individual nodes. A connection is between two ports on two nodes — neither node should own it. The flat connection list avoids duplication and provides a single source of truth for all wiring.

`root` is `Option<Node>` because the empty program is valid and produces no network.

```rust
/// Unique identifier for a node instance within a network.
pub type NodeId = u64;

/// Port direction.
pub enum PortDirection {
    In,
    Out,
    InOut,
}

/// A named, directed port on a node.
pub struct Port {
    pub name: String,
    pub direction: PortDirection,
}

/// A reference to a specific port on a specific node instance.
pub struct PortRef {
    pub node_id: NodeId,
    pub port_name: String,
}

/// A directed connection between two ports.
pub struct Connection {
    pub from: PortRef,
    pub to: PortRef,
}

/// A node in the network. Leaf nodes represent external components;
/// nodes with non-empty children represent groups (composites).
pub struct Node {
    pub id: NodeId,
    pub name: String,        // from the `let` binding in source, or "root" if anonymous
    pub type_name: String,   // schema name for leaf nodes ("Source"); "group" for groups
    pub ports: Vec<Port>,
    pub children: Vec<Node>, // empty for leaf nodes; members for groups
}

/// A warning produced during program evaluation.
pub struct Warning {
    pub kind: WarningKind,
    pub message: String,
}

pub enum WarningKind {
    UnusedValue,
}

/// The complete output of a Nexus program.
pub struct Network {
    pub root: Option<Node>,          // None for programs that produce no network
    pub connections: Vec<Connection>, // all connections, globally owned
}
```

### Root node

The `root` field is the direct result of the program's final expression — no artificial wrapper is added. What the program returns is what the root is:

| Program ends with         | `root`                                                |
| ------------------------- | ----------------------------------------------------- |
| _(nothing)_               | `None`                                                |
| `node Foo`                | `Some(Node { name: "root", type_name: "Foo", .. })`   |
| `let src = node Foo; src` | `Some(Node { name: "src",  type_name: "Foo", .. })`   |
| `group { ... }`           | `Some(Node { name: "root", type_name: "group", .. })` |
| `let n = group { .. }; n` | `Some(Node { name: "n",    type_name: "group", .. })` |

The root name comes from the `let` binding when the result is an identifier expression; otherwise it defaults to `"root"`.

### Entry point

The `Nexus` struct is the public entry point. All integration layers mirror this interface in their respective language.

```rust
pub struct Nexus;

impl Nexus {
    /// Run a Nexus program from source code.
    pub fn run(source: &str) -> Result<(Network, Vec<Warning>), NexusError> { /* ... */ }

    /// Run a Nexus program from a file path.
    pub fn run_file(path: &str) -> Result<(Network, Vec<Warning>), NexusError> { /* ... */ }
}
```

### Helper methods

```rust
impl Network {
    /// Build a map from NodeId to fully qualified path (e.g., "pipeline/source").
    /// Returns an empty map when root is None.
    pub fn node_paths(&self) -> HashMap<NodeId, String> { /* walk tree */ }

    /// Build a map from NodeId to &Node for O(1) lookup.
    /// Returns an empty map when root is None.
    pub fn node_index(&self) -> HashMap<NodeId, &Node> { /* walk tree */ }
}
```

### Usage

```rust
use nexus::Nexus;

let (network, warnings) = Nexus::run_file("network.nxs")?;

for w in &warnings {
    eprintln!("warning: {}", w.message);
}

if let Some(root) = &network.root {
    // Walk the tree for structure
    for node in &root.children {
        println!("{} (type: {})", node.name, node.type_name);
    }

    // Flat iteration over connections
    let paths = network.node_paths();
    for conn in &network.connections {
        println!("{}/{} -> {}/{}",
            paths[&conn.from.node_id], conn.from.port_name,
            paths[&conn.to.node_id],   conn.to.port_name);
    }
}
```

### Design notes

- `Network` is the canonical output — it owns all data, no references back into the interpreter.
- `NodeId` is deterministic within a single run: IDs are assigned via a monotonic counter in declaration order. Same input always produces the same IDs.
- `Node` is recursive: groups contain child nodes which may themselves be groups.
- Nodes and groups are unified: a group is a `Node` with non-empty `children` and `type_name: "group"`. No separate group type is needed.
- Group boundary ports (declared with `port NAME = expr;` inside a group body) appear in the group node's `ports` vec, just like leaf node ports.
- Connections are flat on `Network`, not per-node. Consumers that need per-group wiring can filter by checking whether both endpoint node IDs are children of a given group.
- Node type declarations (`node type`) are not included in the output — they are Nexus's internal validation schema and have served their purpose by the time the network is produced.
- All types derive `Clone`, `Debug`, `PartialEq` at minimum.

## Layer 2: JSON Serialization

The lowest-friction path for cross-language consumption. All `Network` types derive `serde::Serialize` and `serde::Deserialize`, enabling JSON output with zero additional code.

### CLI usage

```
nexus run network.nxs --output json
nexus run network.nxs --output json --pretty
```

### Output format

```json
{
  "root": {
    "id": 1,
    "name": "pipeline",
    "type_name": "group",
    "ports": [],
    "children": [
      {
        "id": 2,
        "name": "source",
        "type_name": "Source",
        "ports": [{ "name": "output", "direction": "out" }],
        "children": []
      },
      {
        "id": 3,
        "name": "sink",
        "type_name": "Sink",
        "ports": [{ "name": "input", "direction": "in" }],
        "children": []
      }
    ]
  },
  "connections": [
    {
      "from": { "node_id": 2, "port_name": "output" },
      "to": { "node_id": 3, "port_name": "input" }
    }
  ]
}
```

An empty program produces:

```json
{ "root": null, "connections": [] }
```

### Design notes

- JSON is the default serialization target. Other formats (MessagePack, CBOR, etc.) come for free via serde if needed later.
- The JSON schema mirrors the Rust types 1:1 — no transformation layer.
- Tree structure is preserved in JSON. Consumers that want a flat node list can post-process by walking `root` recursively.
- `node_id` values in `connections` refer to the `id` fields in the tree and are stable within a single run.

## Layer 3: C ABI FFI

For embedding Nexus in non-Rust host applications. Exposes the Rust API through C-compatible functions and opaque pointers.

### Shared library

`nexus-rs` builds as both a Rust crate (`rlib`) and a C shared library (`cdylib`). The C header is generated with `cbindgen`.

### API sketch

```c
// Opaque handles
typedef struct NexusNetwork NexusNetwork;
typedef struct NexusNodeIter NexusNodeIter;
typedef struct NexusConnIter NexusConnIter;

// Entry point
NexusNetwork* nexus_run(const char* source, size_t source_len);
NexusNetwork* nexus_run_file(const char* path);
void          nexus_network_free(NexusNetwork* net);

// Error handling
const char*   nexus_last_error(void);

// Root node access (returns NULL for empty programs)
int           nexus_network_has_root(const NexusNetwork* net);
uint64_t      nexus_root_id(const NexusNetwork* net);
const char*   nexus_root_name(const NexusNetwork* net);
const char*   nexus_root_type_name(const NexusNetwork* net);

// Node iteration (depth-first tree walk starting from root)
NexusNodeIter* nexus_nodes(const NexusNetwork* net);
int            nexus_node_next(NexusNodeIter* iter,
                               uint64_t*    out_id,
                               const char** out_name,
                               const char** out_type_name,
                               int*         out_child_count);
void           nexus_node_iter_free(NexusNodeIter* iter);

// Connection iteration
NexusConnIter* nexus_connections(const NexusNetwork* net);
int            nexus_conn_next(NexusConnIter* iter,
                               uint64_t*    out_from_node,
                               const char** out_from_port,
                               uint64_t*    out_to_node,
                               const char** out_to_port);
void           nexus_conn_iter_free(NexusConnIter* iter);

// Convenience: JSON export
char*          nexus_to_json(const NexusNetwork* net, int pretty);
void           nexus_string_free(char* s);
```

### C usage

```c
NexusNetwork* network = nexus_run_file("network.nxs");
if (!network) {
    fprintf(stderr, "Error: %s\n", nexus_last_error());
    return 1;
}

if (nexus_network_has_root(network)) {
    NexusNodeIter* nodes = nexus_nodes(network);
    uint64_t id;
    const char* name;
    const char* type_name;
    int child_count;
    while (nexus_node_next(nodes, &id, &name, &type_name, &child_count)) {
        printf("Node %llu: %s (%s), %d children\n", id, name, type_name, child_count);
    }
    nexus_node_iter_free(nodes);
}

NexusConnIter* conns = nexus_connections(network);
uint64_t from_node, to_node;
const char* from_port;
const char* to_port;
while (nexus_conn_next(conns, &from_node, &from_port, &to_node, &to_port)) {
    printf("%llu.%s -> %llu.%s\n", from_node, from_port, to_node, to_port);
}
nexus_conn_iter_free(conns);

nexus_network_free(network);
```

### Design notes

- Opaque pointers + iterator pattern avoids exposing Rust memory layout across the FFI boundary.
- String ownership: functions returning `const char*` return pointers into Nexus-owned memory (valid until the parent handle is freed). Functions returning `char*` (like `nexus_to_json`) transfer ownership — caller must free with `nexus_string_free`.
- Error handling via thread-local last-error string, checked after any function that returns `NULL`.
- The FFI layer is a thin wrapper around the Rust API — no separate logic.
- The C ABI is the universal baseline. The language-specific integrations below (C++ via `cxx`, Python via PyO3) provide more ergonomic alternatives that bypass the C layer entirely.

## Layer 3b: C++ Integration (cxx)

For C++ consumers, `cxx` provides a safer and more ergonomic alternative to the raw C ABI. Instead of opaque pointers and manual memory management, `cxx` generates a shared type system between Rust and C++ with automatic ownership semantics.

### Bridge definition

```rust
// src/ffi_cxx.rs
#[cxx::bridge(namespace = "nexus")]
mod ffi {
    enum PortDirection {
        In,
        Out,
        InOut,
    }

    struct Port {
        name: String,
        direction: PortDirection,
    }

    struct PortRef {
        node_id: u64,
        port_name: String,
    }

    struct Connection {
        from: PortRef,
        to: PortRef,
    }

    extern "Rust" {
        type Network;

        // Entry point
        fn run(source: &str) -> Result<Box<Network>>;
        fn run_file(path: &str) -> Result<Box<Network>>;

        fn has_root(self: &Network) -> bool;
        fn connections(self: &Network) -> &Vec<Connection>;
        fn to_json(self: &Network, pretty: bool) -> Result<String>;
    }
}
```

### C++ usage

```cpp
#include "nexus/src/ffi_cxx.rs.h"

auto network = nexus::run_file("network.nxs");

for (const auto& conn : network->connections()) {
    std::println("{}.{} -> {}.{}", conn.from.node_id, conn.from.port_name, conn.to.node_id, conn.to.port_name);
}
```

### Design notes

- `cxx` generates both Rust and C++ code from the bridge definition — no manual header writing.
- Shared structs (`Port`, `Connection`, etc.) are value types on both sides with zero-cost conversion.
- `String` maps to `rust::String` in C++ with automatic conversion to/from `std::string`.
- `Vec<T>` maps to `rust::Vec<T>` with iterator support.
- `Result<T>` maps to C++ exceptions — errors from the interpreter surface as `rust::Error`.
- Ownership is explicit: `Box<Network>` is a C++ `rust::Box` with move semantics.
- No manual memory management (`free` calls) — RAII on the C++ side.
- `cxx` does not support recursive types directly. `Node` (which contains `Vec<Node>`) needs a workaround: expose the tree via accessor methods that return opaque references, or flatten for the C++ bridge.

## Layer 3c: Python Integration (PyO3)

For Python consumers, PyO3 provides native Python bindings directly from Rust — no C layer needed. The result is a Python package (`nexus`) with idiomatic Python types.

### Rust bindings

```rust
// src/ffi_python.rs
use pyo3::prelude::*;

#[pyclass]
#[derive(Clone)]
struct Node {
    #[pyo3(get)]
    id: u64,
    #[pyo3(get)]
    name: String,
    #[pyo3(get)]
    type_name: String,
    #[pyo3(get)]
    ports: Vec<Port>,
    #[pyo3(get)]
    children: Vec<Node>,
}

#[pyclass]
#[derive(Clone)]
struct Port {
    #[pyo3(get)]
    name: String,
    #[pyo3(get)]
    direction: String,  // "in", "out", or "inout"
}

#[pyclass]
#[derive(Clone)]
struct PortRef {
    #[pyo3(get)]
    node_id: u64,
    #[pyo3(get)]
    port_name: String,
}

#[pyclass]
#[derive(Clone)]
struct Connection {
    #[pyo3(get)]
    from_port: PortRef,
    #[pyo3(get)]
    to_port: PortRef,
}

#[pyclass]
struct Network {
    inner: nexus::Network,
}

#[pymethods]
impl Network {
    #[getter]
    fn root(&self) -> Option<Node> { /* convert from inner */ }

    #[getter]
    fn connections(&self) -> Vec<Connection> { /* convert from inner */ }

    fn node_paths(&self) -> HashMap<u64, String> { /* delegate to inner */ }

    fn to_json(&self, pretty: Option<bool>) -> PyResult<String> { /* serde */ }

    fn __repr__(&self) -> String { /* human-readable summary */ }
}

#[pyfunction]
fn run(source: &str) -> PyResult<Network> { /* ... */ }

#[pyfunction]
fn run_file(path: &str) -> PyResult<Network> { /* ... */ }

#[pymodule]
fn nexus_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(run, m)?)?;
    m.add_function(wrap_pyfunction!(run_file, m)?)?;
    m.add_class::<Network>()?;
    Ok(())
}
```

### Python usage

```python
import nexus

network = nexus.run_file("network.nxs")

if network.root is not None:
    # Walk the tree
    for node in network.root.children:
        print(f"{node.name} ({node.type_name})")

# Flat connection list
paths = network.node_paths()
for conn in network.connections:
    print(f"{paths[conn.from_port.node_id]}/{conn.from_port.port_name}"
          f" -> "
          f"{paths[conn.to_port.node_id]}/{conn.to_port.port_name}")

# JSON export
print(network.to_json(pretty=True))
```

### Distribution

Built with `maturin`, published to PyPI as a wheel:

```
maturin develop    # install locally for development
maturin build      # build wheel
maturin publish    # publish to PyPI
```

### Design notes

- PyO3 compiles to a native Python extension module (`.so`/`.pyd`) — no runtime dependency on Rust.
- Python types are thin wrappers around the Rust types, converted at the boundary.
- `PortDirection` is exposed as a string (`"in"`/`"out"`/`"inout"`) rather than an enum for Python ergonomics. An `enum.Enum` subclass is an alternative if stricter typing is desired.
- `#[pyo3(get)]` provides zero-boilerplate read-only attribute access.
- Interpreter errors map to Python exceptions (`nexus.NexusError`).
- `root` is `None` in Python when the program produces no network (empty program).
- `maturin` handles the build toolchain — Cargo + Python packaging in one step.
- Recursive `Node` works naturally in PyO3 since Python objects are reference-counted.

## Consumer Language Summary

| Language | Integration     | Mechanism             | Ergonomics | Build dependency       |
| -------- | --------------- | --------------------- | ---------- | ---------------------- |
| Rust     | Layer 1 (API)   | Direct crate dep      | Native     | None                   |
| C++      | Layer 3b (cxx)  | Generated bridge      | High       | `cxx`, `cxxbridge-cmd` |
| C        | Layer 3 (FFI)   | `cbindgen` + `cdylib` | Low        | `cbindgen`             |
| Python   | Layer 3c (PyO3) | Native extension      | High       | `pyo3`, `maturin`      |
| Go       | Layer 3 (FFI)   | `cgo` + C header      | Medium     | `cbindgen`             |
| Node.js  | Layer 3 (FFI)   | `ffi-napi`            | Medium     | `cbindgen`             |
| C#/.NET  | Layer 3 (FFI)   | P/Invoke              | Medium     | `cbindgen`             |
| Any      | Layer 2 (JSON)  | Parse stdout/file     | Varies     | None                   |

## Layer 4: Graph Visualization (DOT)

Secondary output format for debugging and documentation. Exports the network as a Graphviz DOT file.

### CLI usage

```
nexus run network.nxs --output dot
nexus run network.nxs --output dot | dot -Tpng -o network.png
```

### Output format

```dot
digraph network {
    rankdir=LR;
    node [shape=record];

    node_2 [label="{<output>output | source: Source}"];
    node_3 [label="{<input>input | sink: Sink}"];

    node_2:output -> node_3:input;
}
```

### Design notes

- Groups map to DOT `subgraph cluster_*` blocks, giving visual containment.
- Port names become record fields, so edges can target specific ports.
- DOT export is a pure formatter on `Network` — no interpreter involvement.
- This is a debugging/documentation tool, not a primary integration path.

## Layer 4b: Mermaid Diagram Output

Mermaid renders natively in GitHub/GitLab markdown, Notion, Confluence, and most modern documentation tools — no external tooling or server required.

### CLI usage

```
nexus run network.nxs --output mermaid
```

### Output format

```mermaid
graph LR
    subgraph pipeline
        source["Source"]
        converter["Converter"]
        sink["Sink"]

        source -- "output → input" --> converter
        converter -- "output → input" --> sink
    end
```

### Design notes

- Groups map to Mermaid `subgraph` blocks.
- Edge labels carry port names (`"output → input"`) since Mermaid nodes don't have port anchors like DOT records.
- Mermaid's main advantage over DOT is zero-dependency rendering in markdown — paste the output into a README and it renders.
- Limited styling compared to DOT/PlantUML, but sufficient for network topology visualization.

## Layer 4c: PlantUML Component Diagram Output

PlantUML produces richer component diagrams with explicit port representations and more layout control. Better suited for formal documentation or complex networks where DOT/Mermaid become hard to read.

### CLI usage

```
nexus run network.nxs --output plantuml
```

### Output format

```plantuml
@startuml
component "source: Source" as node_2 {
    portout " " as node_2_output
}

component "converter: Converter" as node_3 {
    portin  " " as node_3_input
    portout " " as node_3_output
}

component "sink: Sink" as node_4 {
    portin  " " as node_4_input
}

node_2_output --> node_3_input
node_3_output --> node_4_input
@enduml
```

### Design notes

- PlantUML `component` blocks map naturally to leaf nodes; `portin`/`portout` map to declared ports with direction.
- Groups map to PlantUML `package` or `rectangle` blocks for visual containment.
- Requires a PlantUML renderer (Java-based server, or online at plantuml.com) — heavier dependency than Mermaid or DOT.
- Richer styling and layout options: colors, stereotypes, notes, and custom skins.
- Best suited for formal documentation or complex hierarchical networks.

## Implementation Order

| Step | Feature                    | Depends on | Complexity |
| ---- | -------------------------- | ---------- | ---------- |
| 1    | `Network` types            | DSL interp | Small      |
| 2    | Interpreter → API bridge   | Step 1     | Medium     |
| 3    | JSON serialization (serde) | Step 1     | Small      |
| 4    | CLI `--output` flag        | Steps 2-3  | Small      |
| 5    | DOT formatter              | Step 1     | Small      |
| 6    | Mermaid formatter          | Step 1     | Small      |
| 7    | PlantUML formatter         | Step 1     | Small      |
| 8    | C ABI FFI + `cbindgen`     | Step 2     | Medium     |
| 9    | C++ bridge (cxx)           | Step 2     | Medium     |
| 10   | Python bindings (PyO3)     | Step 2     | Medium     |

Steps 1-4 are the practical starting point. Steps 5-7 are visualization formatters — all independent, all small, implement as needed. Steps 8-10 are independent of each other and deferred until a consumer in that language exists.

## Resolved Questions

### R1: Stable node IDs across runs

`NodeId` is deterministic within a single run. IDs are assigned via a monotonic counter in declaration order. Same input always produces the same IDs, so connection endpoints are stable given identical source.

### R2: CLI design

Deferred. The CLI binary is a debugging tool for now. Output format flags will be designed when the binary becomes user-facing.

### R3: Root node

The `root` field is `Option<Node>`. It is set to the direct result of the program's final expression — no artificial wrapper is added. An empty program produces `root: None`. A program ending with a node or group expression produces `root: Some(...)` where the root name comes from the `let` binding if the result is a named identifier, or `"root"` if anonymous.
