# AGENTS.md

This file provides guidance to WARP (warp.dev) when working with code in this repository.

## Project Overview

Athena is a database gateway that proxies and executes queries across Athena/ScyllaDB, PostgreSQL, and Supabase backends. It exposes a unified REST API for executing SQL, gateway operations (fetch/insert/update/delete), and pipeline-based ETL operations.

### Architecture

Three-tier architecture:

1. **API Layer** (`src/api/`): Actix Web Core Service that validates headers, shapes requests, and emits standard response envelopes
2. **Driver Layer** (`src/drivers/`): Router logic that hands off translated queries to PostgreSQL (SQLx), ScyllaDB (native CQL), or Supabase (REST/RPC)
3. **Database Backends**: Actual database connections

Request flow: Client → API Layer (validate X-Athena-Client, resolve backend) → Driver Layer (translate to SQL/CQL) → Database Backend → Response envelope

### Key Components

- **Main API server**: Rust (`athena_rs`) with Actix Web framework
- **Web Explorer UI**: Next.js app in `apps/web/` for visual database interaction
- **TypeScript CLI**: Tool in `cli/` for generating SQL/CQL from Supabase schema metadata
- **Client SDK**: `AthenaClient` in `src/client/` with builder pattern for programmatic access

## Common Development Commands

### Rust API Server

- **Build**: `cargo build --release`
- **Run tests** (requires Supabase): `cargo test --verbose`
- **Lint/Format**: `cargo fmt --check`, `cargo clippy --all-targets --all-features`
- **Start server**: `cargo run -- --api-only` (or `cargo run -- server`)
- **Run with CDC only**: `cargo run -- --cdc-only` (when `cdc` feature enabled)
- **Check README sync**: `cargo rdme --check`

### Rust CLI helpers (athena_rs binary)

- **Global overrides**: `--config/--config-path <PATH>`, `--pipelines <PATH>`, `--port <PORT>`, `--api-only` (same as `server`).
- **Config search order**: `%APPDATA%/athena`, `%LOCALAPPDATA%/athena`, `%USERPROFILE%/.athena`, `$XDG_CONFIG_HOME/athena`, `$HOME/.config/athena`, `$HOME/.athena`, `~/Library/Application Support/athena`, then CWD. Missing configs are auto-copied from the bundled default into the first writable location.
- **Server**: `cargo run -- --api-only` or `cargo run -- server` to boot Actix with overrides above.
- **Pipeline runner**: `cargo run -- pipeline --client <NAME> --pipeline <NAME> --payload-json <JSON>` (or `--payload-file`). Uses definitions from `config/pipelines.yaml`.
- **Clients registry**: `cargo run -- clients list|add|remove|set-default` (mutates `config/clients.json`).
- **Gateway fetch proxy**: `cargo run -- fetch --client <NAME> --body-json <JSON>` (or `--body-file`) with optional `--url`.
- **Diagnostics**: `cargo run -- diag` prints OS/CPU/locale metadata; `cargo run -- version` prints build version.

### TypeScript CLI

- **Setup**: `cd cli && npm install && npm run build`
- **Invoke after build**: `npm start -- <command> [flags]` (or `node dist/main.js <command> [flags]`). Default command is `seed` when none is provided.
- **Seed (default)**: `npm start -- seed --sql=<PATH> --cql=<PATH> --if-not-exist-table --include-constraints-sql` plus Supabase overrides (`--supabase-url`, `--supabase-key`, `--supabase-config=<PATH>`, `--db-config-output=<PATH>`).
- **Constraints only**: `npm start -- constraints --supabase-url=<URL> --supabase-key=<KEY>` (same Supabase overrides as seed).
- **Table registry**: `npm start -- table-registry --registry-schema-path=<PATH> --registry-path=<PATH>` to emit Drizzle registry files.
- **Help**: `npm start -- --help` or `npm start -- help`. Supabase config also respects `ATHENA_SUPABASE_URL`, `ATHENA_SUPABASE_ANON_KEY`, `SUPABASE_URL`, `SUPABASE_ANON_KEY`.

### Web Explorer UI

- **Dev server**: `cd apps/web && npm run dev`
- **Build**: `cd apps/web && npm run build`
- **Lint**: `cd apps/web && npm run lint`
- **Env for local UI**: set `NEXT_PUBLIC_ATHENA_BASE_URL` (defaults to `https://athena-db.com`), `NEXT_PUBLIC_ATHENA_CLIENT`, `NEXT_PUBLIC_ATHENA_USER_ID`, `NEXT_PUBLIC_ATHENA_COMPANY_ID`, `NEXT_PUBLIC_ATHENA_ORGANIZATION_ID` so the datagrid can call schema and gateway endpoints.

## Configuration

Runtime configuration is driven by `config.yaml`:

- **postgres_clients**: Map of logical client names to Postgres connection URIs (use `${ENV_VAR}` tokens for credentials)
- **gateway.force_camel_case_to_snake_case**: Toggle payload normalization
- **api.port**: Server port (default 4052)
- **Cache settings**: Redis write-through with TTL

Pipeline definitions in `config/pipelines.yaml` can be referenced by name in API requests (`POST /pipelines`) or CLI commands.

The `X-Athena-Client` header selects the Postgres pool and drives routing throughout the system. When the client resolves to `custom_supabase`, include `x-supabase-url` and `x-supabase-key` headers.

**Config search and bootstrap**: The loader checks `%APPDATA%/athena`, `%LOCALAPPDATA%/athena`, `%USERPROFILE%/.athena`, `$XDG_CONFIG_HOME/athena`, `$HOME/.config/athena`, `$HOME/.athena`, `~/Library/Application Support/athena`, then the current directory. If nothing is found, the bundled `config.yaml` is copied into the first writable location.

**Client catalog workflow**: Config-backed clients sync into the `athena_logging.athena_clients` table on startup (deduplicated by name). Request-time routing uses the in-memory registry; the catalog is the source of truth for persistence, while `config.yaml` remains required for bootstrap and the logging/auth databases.

## Code Organization

### Core Library Structure

- `src/main.rs`: Binary entry, Actix server startup, routing configuration
- `src/lib.rs`: Library exports, `AppState` struct with shared resources (cache, pg_registry, etc.)
- `src/api/`: All REST endpoints (gateway/\*, pipelines, schema, registry, health)
- `src/drivers/`: Database driver implementations
  - `postgresql/`: SQLx-managed Postgres pools
  - `scylla/`: Native Cassandra/Scylla CQL driver
  - `supabase/`: Supabase REST/RPC client with health tracking
- `src/client/`: `AthenaClient` SDK with builder pattern for unified query interface
- `src/pipeline/`: ETL pipeline execution (source → transform → sink)
- `src/data/`: Data access helpers and Supabase client utilities
- `src/config.rs`: Config loading with multi-platform search paths
- `src/router.rs`: HTTP routing helpers and target URL determination

### Important Patterns

**Standard Response Envelope**: All API endpoints return `{ "status": "success" | "error", "message": "...", "data": {...}?, "error": {...}? }`. Use `api_success`, `internal_error`, and related helpers from `api::response`. Every endpoint must document and support at least one **2xx** (success) and one **5xx** (server error) response; **3xx** (redirect) is optional where applicable. The OpenAPI spec and default error handlers (404, 5xx) enforce this contract.

**Parameterized SQL**: When building SQL in API handlers, use driver parameter binding instead of string interpolation to prevent SQL injection. For example: `session.query("SELECT * FROM events WHERE id = ?", (id,))` rather than `format!("SELECT * FROM events WHERE id = {}", id)`.

**Post-Processing**: Gateway endpoints (`/gateway/fetch`, `/gateway/update`) support optional `group_by`, `time_granularity`, `aggregation_column`, `aggregation_strategy`, and `aggregation_dedup` fields. Responses include `post_processing.grouped` array when grouping is enabled.

**Health Tracking**: The Supabase driver wraps the client in a circuit breaker. Use `is_offline()`, `force_offline()`, and `reset_health()` methods on health-aware clients.

## WebSocket Gateway

`src/wss/` serves `openapi-wss.yaml` and `/wss/info`; the transport mirrors gateway actions over WebSocket (`gateway.fetch`, `gateway.insert`, `gateway.update`, `gateway.delete`, `gateway.query`). Use the spec when prototyping multiplexed clients.

## Feature Flags

- **cdc** (default): Includes WebSocket server and CDC subcommand. Disable with `--no-default-features` for smaller binary without WebSocket support.

## Testing

Tests are in `tests/` directory. The CI workflow (`.github/workflows/rust.yml`) starts Supabase before running tests. Use `cargo test` to run all tests.

## Docker

- Pre-pull optional cache image: `docker pull redis:8.2.4-alpine3.22`
- Build and run with Compose: `docker compose up --build`
- Server listens on http://localhost:4052; caching activates when `ATHENA_REDIS_URL=redis://redis:6379`
- Stop with `docker compose down` (preserves redis-data volume)

## Cursor Cloud specific instructions

### Prerequisites (already installed in snapshots)

- **Rust >= 1.85** (edition 2024 requirement). The VM ships with 1.83; run `rustup update stable && rustup default stable` if needed.
- **libssl-dev** and **pkg-config** for the `openssl-sys` crate.
- **Docker** (Docker-in-Docker with `fuse-overlayfs` storage driver and `iptables-legacy`).

### Starting services

1. **PostgreSQL** (required for the Rust API):

   ```
   sudo dockerd &>/tmp/dockerd.log &
   sleep 3
   sudo docker compose up -d postgres
   ```

   The `athena_logging` database is auto-provisioned via `sql/provision.sql`.

2. **Rust API Server** (port 4052):

   ```
   POSTGRES_ATHENA_LOGGING_URI="postgres://athena:athena@localhost:5433/athena_logging" cargo run -- --api-only
   ```

   The server logs warnings for missing remote Postgres clients (`railway_direct`, etc.)—this is expected in local dev since those are external databases. Only `athena_logging` is needed locally.

3. **Web Explorer UI** (port 3001):
   ```
   cd apps/web && NEXT_PUBLIC_ATHENA_BASE_URL=http://localhost:4052 NEXT_PUBLIC_ATHENA_CLIENT=athena_logging npm run dev -- -p 3001
   ```
   Without `NEXT_PUBLIC_ATHENA_CLIENT` the UI starts in a "No client connected" state and you must manually set the client via the sidebar.

### Gotchas

- The `config.yaml` references many Postgres clients via `${ENV_VAR}` tokens (e.g., `POSTGRES_RAILWAY_DIRECT_URI`). These are production/remote databases. The server tolerates missing clients and continues with just `athena_logging`.
- `cargo test` requires a running Supabase instance (see CI workflow in `.github/workflows/rust.yml` for the `SUPABASE_URL` and `SUPABASE_KEY` values used in CI).
- The TypeScript CLI (`cli/`) is missing its `tsconfig.json`, so `npm run build` fails. The CLI is optional and requires Supabase credentials.
- The Web Explorer shows "No tables found" until a valid `X-Athena-Client` is configured in the UI settings (click "Set key" in the sidebar).
