📚 Syntax Knowledge Base

Overview

The Syntax Knowledge Base (Syntax KB) is a persistent, thread-safe storage system for syntax definitions, protocols, and schemas. It enables:

• 🔍 Syntax Discovery - Agents can query available protocols and formats
• 🤝 Inter-Agent Communication - Shared understanding of data structures
• 📖 Self-Documentation - Built-in specifications with examples
• 🔧 Extensibility - Add custom syntaxes at runtime
• 💾 Persistence - Stored in ~/.aethershell/syntax_kb.json

Built-in Syntaxes

AgenticBinary Protocol (ab)

JSON-RPC (jsonrpc)

HTTP (http)

Syntax KB Functions

Retrieving Syntax

// Get specific syntax by ID
ab_spec = syntax_get("ab")

// Access fields
print(ab_spec.id)             // "ab"
print(ab_spec.name)           // "AgenticBinary Protocol"
print(ab_spec.category)       // "protocol"
print(ab_spec.specification)  // Full spec...
print(ab_spec.examples)       // Array of examples

// Pipeline mode
"ab" | syntax_get | keys  // List all fields

Listing Syntaxes

// List all syntax IDs
all_ids = syntax_list()
print(all_ids)  // ["ab", "jsonrpc", "http"]

// List by category
protocols = syntax_list("protocol")
print(protocols)  // ["ab", "jsonrpc", "http"]

// Pipeline mode
"protocol" | syntax_list  // Filter by category

Searching Syntax

// Keyword search
binary_results = syntax_search("binary")
print(binary_results)  // ["ab"]

agent_results = syntax_search("agent")
print(agent_results)  // ["ab"]

// Pipeline mode
"protocol" | syntax_search

Adding Custom Syntax

// Define new syntax
my_protocol = {
    id: "myproto",
    name: "My Custom Protocol",
    category: "protocol",
    specification: "Custom binary protocol for domain-specific tasks...",
    examples: [
        "Example 1: Header format",
        "Example 2: Payload encoding"
    ]
}

// Add to knowledge base
syntax_add(my_protocol)

// Verify it was added
retrieved = syntax_get("myproto")
print(retrieved.name)  // "My Custom Protocol"

// Pipeline mode
{id: "test", name: "Test"} | syntax_add

Listing Categories

// Get all available categories
categories = syntax_categories()
print(categories)
// ["protocol", "grammar", "format", "encoding", "schema"]

AgenticBinary in Syntax KB

Learning the Protocol

// Agent retrieves AB protocol specification
spec = syntax_get("ab")

// Agent learns the protocol structure
print("Protocol: ${spec.name}")
print("Category: ${spec.category}")
print("Specification:")
print(spec.specification)

// Agent studies examples
spec.examples | map(fn(ex) => print(ex))

Protocol Specification Fields

ab_spec = syntax_get("ab")

// Core fields
ab_spec.id               // "ab"
ab_spec.name             // "AgenticBinary Protocol"
ab_spec.category         // "protocol"

// Detailed information
ab_spec.specification    // Full protocol description
ab_spec.examples         // Usage examples

// Protocol details (within specification)
// - Header format: 0bVVTTCCCC
// - Message types: command, query, response, event
// - 16 semantic opcodes
// - Varint payload encoding

Multi-Agent Usage Pattern

Phase 1: Protocol Discovery

// Worker discovers available protocols
protocols = syntax_list("protocol")
print("Available protocols: ${protocols}")

// Worker selects AgenticBinary
spec = syntax_get("ab")
print("Using: ${spec.name}")

Phase 2: Protocol Learning

// Worker learns protocol specification
learn_msg = ab_encode("command", "learn", spec.specification)

// Worker acknowledges learning
ack = ab_encode("response", "ack", "learned:ab_protocol")

Phase 3: Communication

// Now workers can communicate using AgenticBinary
ping = ab_encode("command", "ping", "worker1:ready")
ack = ab_encode("response", "ack", "coordinator:acknowledged")
task = ab_encode("command", "delegate", "analyze_logs")
result = ab_encode("response", "data", "complete:analysis.json")

Custom Syntax Examples

Domain-Specific Format

// Define custom data format
csv_format = {
    id: "csv",
    name: "CSV Format",
    category: "format",
    specification: "Comma-Separated Values format...",
    examples: [
        "name,age,city\nAlice,30,NYC\nBob,25,LA"
    ]
}
syntax_add(csv_format)

Custom Grammar

// Define custom grammar
my_grammar = {
    id: "mygrammar",
    name: "My DSL Grammar",
    category: "grammar",
    specification: "expr := term (('+' | '-') term)*...",
    examples: [
        "1 + 2 * 3",
        "(x + y) / z"
    ]
}
syntax_add(my_grammar)

Encoding Scheme

// Define custom encoding
base58_encoding = {
    id: "base58",
    name: "Base58 Encoding",
    category: "encoding",
    specification: "Bitcoin-style Base58 encoding...",
    examples: [
        "123456 => BukQL",
        "hello => Cn8eVZg"
    ]
}
syntax_add(base58_encoding)

Syntax KB Storage

Persistence Location

# Unix/Linux/Mac
~/.aethershell/syntax_kb.json

# Windows
%USERPROFILE%\.aethershell\syntax_kb.json

Storage Format (JSON)

{
  "version": 1,
  "syntaxes": [
    {
      "id": "ab",
      "name": "AgenticBinary Protocol",
      "category": "protocol",
      "specification": "...",
      "examples": [...]
    },
    {
      "id": "jsonrpc",
      "name": "JSON-RPC 2.0",
      "category": "protocol",
      "specification": "...",
      "examples": [...]
    }
  ]
}

Thread Safety

The Syntax KB uses:

• 🔒 OnceLock<Mutex<SyntaxKB>> for thread-safe singleton
• 🔄 Automatic file synchronization on updates
• 💾 Atomic writes to prevent corruption

Categories

Best Practices

Syntax Entry Guidelines

Multi-Agent Communication

• 🔹 Use syntax_get("ab") to ensure agents speak the same protocol
• 🔹 Share custom syntaxes via LEARN opcode
• 🔹 Query available syntaxes with syntax_list() for discovery
• 🔹 Search by keyword using syntax_search()

Performance Tips

• ✅ Cache retrieved syntaxes locally in agent memory
• ✅ Use syntax_list(category) to filter results
• ✅ Avoid repeated syntax_add() calls for same syntax

Complete Example

See examples/12_syntax_kb.ae for a comprehensive demonstration including:

• ✅ Retrieving built-in syntaxes
• ✅ Searching by keyword
• ✅ AgenticBinary encode/decode
• ✅ Adding custom syntax
• ✅ Multi-agent workflow simulation

# Run the example
ae examples/12_syntax_kb.ae