# WarpDrive Plugin System - Capabilities Verification

## Status: ✅ VERIFIED (2025-01-17)

This document contains verified findings from codebase analysis about WarpDrive's WASM plugin system capabilities.

---

## Executive Summary

### ✅ Confirmed Capabilities
1. Plugins can **access request context** (method, path, headers, body)
2. Plugins can **block requests** via return codes
3. Plugins can **log to host**
4. Plugins can **control WebSocket connections** (send/broadcast/close)
5. Plugins run in **sandboxed environment** with fuel limits

### ❌ Missing Capabilities (Not Implemented)
1. **Cannot modify request headers** before forwarding
2. **Cannot intercept responses** (no response-side hooks)
3. **Cannot store persistent state** (no key-value host functions)
4. **Cannot make HTTP requests** (no network access from sandbox)
5. **No signal-based config reload** (SIGTERM/SIGINT only for shutdown)

---

## Detailed Findings

### 1. Request Header Modification ❌ NOT SUPPORTED

**Evidence:**
- Plugins receive headers as **read-only string** parameter
- No `host_set_request_header()` function exists in `src/middleware/wasm_plugin/host.rs`
- Plugin return value is `i32` (status code), not a structure with modified headers
- Request modification happens in `upstream_request_filter()` but **plugins are not invoked there**

**Code Reference:**
```rust
// src/middleware/wasm_plugin/runtime.rs:120-163
pub async fn on_http_request(
    &mut self,
    method: &str,
    path: &str,
    headers: &str,  // <-- Read-only string, not mutable
    body: &[u8],
) -> Result<i32>
```

**Current Flow:**
```
Request → Middleware Chain → Plugin (read-only) → Upstream Request Filter → Upstream
                              ↓
                         Return 0 or 1
                       (allow or block)
```

**Workaround:** Not possible without code changes

**Fix Required:**
- Add `host_set_request_header(name: string, value: string) -> i32` host function
- Pass mutable header map to plugin via shared memory
- Plugin writes modified headers, host reads them back

---

### 2. Response-Side Hooks ❌ NOT IMPLEMENTED

**Evidence:**
- Grep for `wasm_on_response` returns **zero matches**
- Only request-side entry points exist:
  - `wasm_on_request` (HTTP)
  - `wasm_on_event` (WebSocket)
  - `wasm_on_grpc_call` (gRPC)
- `response_filter()` in proxy handler **does not call plugins**

**Code Reference:**
```rust
// src/proxy/handler.rs:305-317
async fn response_filter(
    &self,
    session: &mut Session,
    upstream_response: &mut ResponseHeader,
    ctx: &mut Self::CTX,
) -> Result<()> {
    self.middleware
        .apply_response_filters(session, upstream_response, ctx)
        .await
    // ^ No plugin invocation here
}
```

**Impact:**
- Cannot inject `Set-Cookie` headers
- Cannot modify response status codes
- Cannot transform response bodies
- Cannot implement response-based auth (JWT refresh)

**Workaround:** Not possible without code changes

**Fix Required:**
- Add `wasm_on_response()` entry point to plugin ABI
- Call plugin from `response_filter()` with upstream response
- Allow plugin to return modified headers/status

---

### 3. Plugin State Storage ❌ NOT AVAILABLE

**Evidence:**
- No `host_kv_get` or `host_kv_set` functions in host.rs
- Only 4 host functions exported:
  1. `host_log` - Logging
  2. `host_send` - WebSocket send
  3. `host_broadcast` - WebSocket broadcast
  4. `host_close` - WebSocket close
- Plugin memory is ephemeral (reset per request)

**Code Reference:**
```rust
// src/middleware/wasm_plugin/host.rs:10-104
pub fn define_host_functions(linker: &mut Linker<PluginState>) -> Result<()> {
    linker.func_wrap("warpdrive", "host_log", ...)?;
    linker.func_wrap("warpdrive", "host_send", ...)?;
    linker.func_wrap("warpdrive", "host_broadcast", ...)?;
    linker.func_wrap("warpdrive", "host_close", ...)?;
    Ok(())
}
// ^ Only 4 functions, no KV storage
```

**Impact:**
- Cannot cache JWT validation results
- Cannot implement rate limiting in plugin
- Cannot track session state
- Must re-validate on every request (expensive)

**Workaround:** Plugin could theoretically call external Redis via custom host function, but:
- No `host_http_call` exists
- Cannot make network requests from sandbox

**Fix Required:**
- Add `host_kv_get(key: string) -> string` host function
- Add `host_kv_set(key: string, value: string, ttl_secs: i32) -> i32` host function
- Backend: Use DashMap or Redis for plugin-scoped storage

---

### 4. HTTP Requests from Plugins ❌ NOT POSSIBLE

**Evidence:**
- No `host_http_call` or similar function
- WASM sandbox has **no network access** by design
- Cannot verify tokens with external auth server
- Cannot fetch data from APIs

**Code Reference:**
- Confirmed by absence in `src/middleware/wasm_plugin/host.rs` (only 4 functions)
- Wasmtime config explicitly disables networking features

**Impact:**
- Cannot validate JWT with JWKS endpoint
- Cannot call OAuth introspection endpoints
- Cannot fetch user permissions from API
- Cannot implement webhook plugins

**Workaround:** Not possible without adding host function

**Fix Required:**
- Add `host_http_call(method: string, url: string, headers: string, body: bytes) -> i32`
- Return handle to async response
- Add `host_http_read_response(handle: i32) -> bytes`

---

### 5. Signal Handling ❌ NO CONFIG RELOAD SUPPORT

**Evidence from `src/main.rs` and `src/process/manager.rs`:**

**Signals Handled:**
- `SIGTERM` - Forwarded to supervised processes, triggers shutdown
- `SIGINT` - Forwarded to supervised processes, triggers shutdown

**Code Reference:**
```rust
// src/process/manager.rs:515-543
async fn handle_signals(&self) {
    let mut sigterm = match signal(SignalKind::terminate()) {
        Ok(s) => s,
        Err(e) => {
            error!("Failed to setup SIGTERM handler: {}", e);
            return;
        }
    };

    let mut sigint = match signal(SignalKind::interrupt()) {
        Ok(s) => s,
        Err(e) => {
            error!("Failed to setup SIGINT handler: {}", e);
            return;
        }
    };

    tokio::select! {
        _ = sigterm.recv() => {
            info!("Received SIGTERM, forwarding to all processes");
            self.forward_signal_to_all(SignalKind::terminate()).await;
        }
        _ = sigint.recv() => {
            info!("Received SIGINT, forwarding to all processes");
            self.forward_signal_to_all(SignalKind::interrupt()).await;
        }
    }
}
```

**What Happens:**
1. WarpDrive receives SIGTERM/SIGINT
2. Signals forwarded to supervised child processes
3. WarpDrive enters **shutdown sequence**:
   - Execute `pre_shutdown` hooks
   - Stop all managed processes
   - Execute `post_shutdown` hooks
   - Exit

**No Reload Behavior:**
- No `SIGHUP` handler for config reload
- No `SIGUSR1` handler for graceful restart
- No `SIGUSR2` handler for binary upgrade

**Comparison with Aralez:**
- Aralez: File watcher (inotify) → hot reload config without restart
- WarpDrive: Config changes require **new container deployment**

**This is by design** for immutable infrastructure:
- Config = part of container image or volume mount
- Changes = deploy new container (blue-green)
- No runtime mutation = no config drift

**TLS Certificate Rotation:**
- Certs stored in volume mount: `/app/certs:ro`
- Rotation requires: New container or restart
- No `SIGHUP` to reload certs without downtime

---

## Plugin Use Cases Analysis

### ✅ What Plugins CAN Do Today

#### 1. WebSocket Message Filtering
```rust
// Plugin can inspect and block messages
wasm_on_event(event_type: CONNECT, conn_id, payload) -> {
    if !is_authorized(conn_id) {
        host_close(conn_id, 1008, "Unauthorized");
        return 1; // Block
    }
    return 0; // Allow
}
```

#### 2. Request Logging/Auditing
```rust
wasm_on_request(method, path, headers, body) -> {
    let user = extract_user_from_headers(headers);
    host_log(INFO, format!("{} {} {}", user, method, path));
    return 0; // Allow
}
```

#### 3. Simple Path-Based Routing
```rust
wasm_on_request(method, path, headers, body) -> {
    if path.starts_with("/admin") && !is_admin(headers) {
        return 403; // Block
    }
    return 0; // Allow
}
```

#### 4. WebSocket Broadcasting
```rust
wasm_on_event(event_type: MESSAGE, conn_id, payload) -> {
    let stream = get_stream_from_payload(payload);
    host_broadcast(stream, payload); // Fan-out
    return 0;
}
```

### ❌ What Plugins CANNOT Do Today

#### 1. JWT Validation (Missing: HTTP calls, KV storage)
```rust
// CANNOT IMPLEMENT - No host_http_call for JWKS fetch
wasm_on_request(method, path, headers, body) -> {
    let token = extract_bearer_token(headers);
    // ❌ Cannot fetch public key from auth server
    // ❌ Cannot cache validation results
    return 401; // Have to block everything or allow everything
}
```

**Workaround:** External auth service before WarpDrive

#### 2. Rate Limiting (Missing: KV storage)
```rust
// CANNOT IMPLEMENT - No host_kv_get/set for counters
wasm_on_request(method, path, headers, body) -> {
    let client_ip = extract_ip(headers);
    // ❌ Cannot track request count per IP
    // ❌ Cannot implement token bucket
    return 0; // No rate limiting possible
}
```

**Workaround:** Use built-in Pingora rate limiting or external service

#### 3. Cookie-Based Auth (Missing: Response hooks)
```rust
// CANNOT IMPLEMENT - No wasm_on_response to set cookies
wasm_on_request(method, path, headers, body) -> {
    let session = validate_session_cookie(headers);
    if session.expired {
        // ❌ Cannot set new cookie in response
        return 401; // Can only block
    }
    return 0;
}
```

**Workaround:** Cookies must be set by upstream app

#### 4. Header Injection (Missing: Header modification)
```rust
// CANNOT IMPLEMENT - Headers are read-only
wasm_on_request(method, path, headers, body) -> {
    // ❌ Cannot add X-User-ID header
    // ❌ Cannot add X-Request-ID for tracing
    return 0; // Pass through unchanged
}
```

**Workaround:** Use route-level header injection in TOML config

---

## Architecture Implications

### Plugin Limitations Summary

| Capability | Status | Reason | Impact |
|------------|--------|--------|--------|
| Read request headers | ✅ Works | String parameter | Can inspect |
| Modify request headers | ❌ Missing | No host function | Cannot inject |
| Block requests | ✅ Works | Return code 1 | Can reject |
| Read response | ❌ Missing | No wasm_on_response | Cannot inspect |
| Modify response | ❌ Missing | No response hook | Cannot transform |
| Store state | ❌ Missing | No host_kv_* | Must be stateless |
| HTTP calls | ❌ Missing | No host_http_call | Cannot fetch data |
| WebSocket send | ✅ Works | host_send | Real-time messaging |
| WebSocket broadcast | ✅ Works | host_broadcast | Fan-out |
| WebSocket close | ✅ Works | host_close | Connection control |

### Recommended Plugin Use Cases (Given Limitations)

#### ✅ Good Fit
- **WebSocket middleware** - Full control over connections
- **Request logging/auditing** - Read-only inspection
- **Simple path-based blocking** - No external data needed
- **Static rule enforcement** - Rules compiled into WASM

#### ⚠️ Possible with Workarounds
- **Auth** - Use external auth service before WarpDrive
- **Rate limiting** - Use Pingora's built-in or external service
- **Header injection** - Use TOML config route headers

#### ❌ Not Feasible
- **Dynamic JWT validation** - Need HTTP calls for JWKS
- **Session management** - Need response cookies
- **Request transformation** - Headers are read-only
- **Response transformation** - No response hooks

---

## Comparison: Built-in vs Plugin Approach

### Aralez (Built-in)
```toml
# Authentication built into core
[auth]
type = "jwt"
secret = "..."
```

**Pros:**
- Works out of the box
- No plugin development needed
- Optimized for performance

**Cons:**
- Limited to what's built-in
- Need to fork/rebuild for customization

### WarpDrive (Plugin)
```toml
[[routes]]
path_prefix = "/api"
upstream = "api"
plugin = "plugins/auth.wasm"
```

**Pros:**
- Extensible without forking
- Load custom logic at runtime
- Sandboxed security

**Cons:**
- **Limited host functions** (verified gaps above)
- Cannot implement full auth without HTTP calls
- Cannot set response headers/cookies
- No state storage

---

## Development Roadmap (Priority Order)

### Phase 1: Request-Side Completeness (High Priority)

#### 1.1 Header Modification
```rust
// Add to host.rs
linker.func_wrap("warpdrive", "host_set_request_header",
    |mut caller: Caller<'_, PluginState>, name_ptr: i32, name_len: i32, value_ptr: i32, value_len: i32| -> i32 {
        // Implementation
    }
)?;
```

**Enables:**
- X-User-ID injection
- X-Request-ID tracing
- Custom auth headers

#### 1.2 Key-Value Storage
```rust
linker.func_wrap("warpdrive", "host_kv_get",
    |mut caller: Caller<'_, PluginState>, key_ptr: i32, key_len: i32| -> i32 {
        // Return pointer to value
    }
)?;

linker.func_wrap("warpdrive", "host_kv_set",
    |mut caller: Caller<'_, PluginState>, key_ptr: i32, key_len: i32, value_ptr: i32, value_len: i32, ttl: i32| -> i32 {
        // Store with expiration
    }
)?;
```

**Enables:**
- JWT validation caching
- Rate limiting counters
- Session state tracking

### Phase 2: External Data Access (Medium Priority)

#### 2.1 HTTP Client
```rust
linker.func_wrap("warpdrive", "host_http_call",
    |mut caller: Caller<'_, PluginState>, method_ptr: i32, url_ptr: i32, headers_ptr: i32, body_ptr: i32| -> i32 {
        // Return request handle (async)
    }
)?;

linker.func_wrap("warpdrive", "host_http_await",
    |mut caller: Caller<'_, PluginState>, handle: i32| -> i32 {
        // Block until response ready
    }
)?;

linker.func_wrap("warpdrive", "host_http_read_response",
    |mut caller: Caller<'_, PluginState>, handle: i32| -> i32 {
        // Return pointer to response
    }
)?;
```

**Enables:**
- JWKS fetching
- OAuth token introspection
- External auth service calls
- Webhook plugins

### Phase 3: Response-Side Hooks (Medium Priority)

#### 3.1 Response Interception
```rust
// Plugin ABI addition
wasm_on_response(
    status_code: i32,
    headers_ptr: i32,
    headers_len: i32,
    body_ptr: i32,
    body_len: i32
) -> i32

// Host function for modification
host_set_response_header(name: string, value: string) -> i32
host_set_response_status(code: i32) -> i32
```

**Enables:**
- Cookie injection (Set-Cookie)
- CORS header injection
- Response transformation
- JWT refresh tokens

### Phase 4: Advanced Features (Low Priority)

#### 4.1 Signal-Based Reload (NOT RECOMMENDED)
- Conflicts with immutable infrastructure philosophy
- Better: Blue-green deployment with new config

#### 4.2 Plugin State Persistence
- Async Redis integration from host
- Shared state across instances

---

## Conclusion

### Current State (v0.3.0)
WarpDrive's plugin system is **functional but limited** to:
- ✅ Request inspection (read-only)
- ✅ Request blocking
- ✅ WebSocket control
- ❌ Cannot modify requests/responses
- ❌ Cannot store state
- ❌ Cannot make HTTP calls

### Architectural Philosophy
The limitations align with **immutable infrastructure**:
- Plugins for **simple** logic (blocking, logging)
- Complex features (auth, rate limiting) via **external services** or **built-in middleware**
- Config changes via **new deployments**, not hot reload

### Recommendation
For Seuros's use case, choose:
1. **Simple filtering** → Use plugins (works today)
2. **JWT auth** → Use external auth proxy (e.g., Aralez) in front
3. **Dynamic routing** → Use TOML config + rolling updates
4. **State management** → Use Redis L2 cache + external auth service

**If you need full auth in plugins:** Implement Phase 1 + Phase 2 from roadmap above.

---

## Example: What a Complete JWT Plugin Would Need

```rust
// Ideal plugin (requires missing features)
wasm_on_request(method, path, headers, body) -> i32 {
    // Extract token
    let token = extract_bearer_token(headers);

    // ❌ MISSING: Check cache
    let cached_result = host_kv_get(format!("jwt:{}", token));
    if cached_result == "valid" {
        // ❌ MISSING: Inject user ID
        host_set_request_header("X-User-ID", "123");
        return 0; // Allow
    }

    // ❌ MISSING: Fetch JWKS
    let jwks_response = host_http_call("GET", "https://auth.example.com/.well-known/jwks.json", "", "");
    host_http_await(jwks_response);
    let jwks = host_http_read_response(jwks_response);

    // Validate token
    if !validate_jwt(token, jwks) {
        return 401; // Block
    }

    // ❌ MISSING: Cache result
    host_kv_set(format!("jwt:{}", token), "valid", 300); // 5min TTL

    // ❌ MISSING: Inject header
    host_set_request_header("X-User-ID", get_user_id(token));

    return 0; // Allow
}
```

**Missing Pieces:**
1. `host_kv_get` / `host_kv_set` - State storage
2. `host_http_call` / `host_http_await` / `host_http_read_response` - HTTP client
3. `host_set_request_header` - Header modification

**Without these:** JWT validation must happen externally (before WarpDrive) or in upstream app.
