# FraiseQL Core Benchmarks

Comprehensive performance benchmarking suite for FraiseQL database adapters.

## Available Benchmarks

### 1. `adapter_comparison` - PostgreSQL vs FraiseQL-Wire

Compares performance between two database adapter implementations:

| Adapter | Implementation | Memory Model | Best For |
|---------|---------------|--------------|----------|
| **PostgresAdapter** | tokio-postgres | Buffered (O(n)) | Small results, transactions, writes |
| **FraiseWireAdapter** | fraiseql-wire | Streaming (O(1)) | Large results, memory-constrained |

**Metrics Measured**:

- ⏱️  Throughput (rows/second)
- 📊 Query latency (total time)
- 🚀 Time-to-first-row
- 💾 Memory usage (via external profiling)
- 🔍 WHERE clause performance
- 📄 Pagination efficiency

### 2. `full_pipeline_comparison` - Complete GraphQL Pipeline

Benchmarks the complete FraiseQL execution pipeline:

1. Database query execution
2. Field projection (selecting only requested fields)
3. Field name transformation (snake_case → camelCase)
4. `__typename` addition
5. GraphQL data envelope wrapping

```bash
cargo bench --bench full_pipeline_comparison --features postgres
```

## Quick Start

### 1. Set Up Test Database

```bash
# Create database
createdb fraiseql_bench

# Load 1M rows of test data (~30-60 seconds)
psql fraiseql_bench < benches/fixtures/setup_bench_data.sql

# Set environment variable
export DATABASE_URL="postgresql:///fraiseql_bench"
# Or with custom connection:
export DATABASE_URL="postgresql://user:pass@localhost:5432/fraiseql_bench"
```

### 2. Run Benchmarks

```bash
# Run all benchmarks (both adapters)
cargo bench --bench adapter_comparison --features "postgres,wire-backend"

# Run only PostgresAdapter benchmarks
cargo bench --bench adapter_comparison --features postgres

# Run only FraiseWireAdapter benchmarks
cargo bench --bench adapter_comparison --features wire-backend

# Run specific benchmark group
cargo bench --bench adapter_comparison -- "10k_rows"
cargo bench --bench adapter_comparison -- "100k_rows"
cargo bench --bench adapter_comparison -- "where_clause"
cargo bench --bench adapter_comparison -- "pagination"
```

### 3. View Results

Results are saved in `target/criterion/`:

```bash
# Open HTML report
open target/criterion/report/index.html

# View specific benchmark
open target/criterion/10k_rows/report/index.html
```

## Benchmark Groups

### Small Queries (10K rows)

Tests performance on typical small-to-medium result sets.

```bash
cargo bench --bench adapter_comparison -- "10k_rows"
```

**Expected Results**:

- **Throughput**: ~300K rows/s (both adapters)
- **Latency**: 30-35ms (PostgresAdapter slightly faster)
- **Memory**: 260KB vs 1.3KB (fraiseql-wire 200x better)

### Medium Queries (100K rows)

Demonstrates memory efficiency differences at scale.

```bash
cargo bench --bench adapter_comparison -- "100k_rows"
```

**Expected Results**:

- **Throughput**: ~300K rows/s (comparable)
- **Latency**: 300-350ms
- **Memory**: 26MB vs 1.3KB (fraiseql-wire 20,000x better) ⭐

### Large Queries (1M rows)

Stress test for extreme result sets.

```bash
cargo bench --bench adapter_comparison -- "1m_rows"
```

**Expected Results**:

- **Throughput**: ~250K rows/s (fraiseql-wire may be faster due to less GC pressure)
- **Latency**: 4-5 seconds
- **Memory**: 260MB vs 1.3KB (fraiseql-wire 200,000x better) ⭐⭐⭐

### WHERE Clause Performance

Tests filtered query performance.

```bash
cargo bench --bench adapter_comparison -- "where_clause"
```

**Filters**: `data->>'status' = 'active'` (~250K matching rows out of 1M)

**Expected Results**:

- **Both adapters**: Comparable performance (PostgreSQL does the filtering)
- **Memory**: fraiseql-wire still uses O(chunk_size), not O(result_size)

### Pagination

Tests repeated small queries (10 pages × 100 rows).

```bash
cargo bench --bench adapter_comparison -- "pagination"
```

**Expected Results**:

- **PostgresAdapter**: Faster (connection reuse, prepared statements)
- **FraiseWireAdapter**: Slightly slower (creates new client per query)

**Note**: FraiseWireAdapter could be optimized with connection pooling in future.

## Memory Profiling

### Using heaptrack (Linux)

```bash
# Install heaptrack
sudo apt install heaptrack heaptrack-gui  # Debian/Ubuntu
sudo pacman -S heaptrack                  # Arch Linux

# Build release binary
cargo build --release --bench adapter_comparison --features "postgres,wire-backend"

# Profile PostgresAdapter (100K rows)
heaptrack target/release/deps/adapter_comparison-* -- "postgres_adapter/100k_rows" --bench

# Profile FraiseWireAdapter (100K rows)
heaptrack target/release/deps/adapter_comparison-* -- "wire_adapter/100k_rows" --bench

# Open GUI to view results
heaptrack_gui heaptrack.adapter_comparison.*.gz
```

### Using Valgrind Massif (Cross-platform)

```bash
# Install valgrind
sudo apt install valgrind  # Linux
brew install valgrind      # macOS (may need rosetta)

# Profile memory usage
valgrind --tool=massif \
    target/release/deps/adapter_comparison-* \
    -- "100k_rows" --bench

# View results
ms_print massif.out.*
```

### Expected Memory Profile

**PostgresAdapter (100K rows)**:

```
Peak memory: ~26 MB
- Result buffer: ~25 MB (all rows buffered)
- Connection overhead: ~1 MB
```

**FraiseWireAdapter (100K rows)**:

```
Peak memory: ~1.3 KB
- Chunk buffer: ~1 KB (1024 rows × ~1 byte per row)
- Stream overhead: ~300 bytes
```

## Interpreting Results

### Criterion Output

```
10k_rows/postgres_adapter/collect_all
                        time:   [32.145 ms 32.456 ms 32.789 ms]
                        thrpt:  [305.15 Kelem/s 308.12 Kelem/s 311.09 Kelem/s]

10k_rows/wire_adapter/stream_collect
                        time:   [33.234 ms 33.567 ms 33.912 ms]
                        thrpt:  [294.82 Kelem/s 297.91 Kelem/s 300.88 Kelem/s]
```

**Reading**:

- `time`: Median query time with confidence interval
- `thrpt`: Throughput in thousands of elements per second
- Lower time = faster
- Higher thrpt = faster

### Comparison Reports

Criterion generates comparison reports when you re-run benchmarks:

```
10k_rows/postgres_adapter vs 10k_rows/wire_adapter
                        time:   [+2.85% +3.42% +4.01%]
                        thrpt:  [-3.85% -3.31% -2.77%]
```

**Reading**:

- `+3.42%` = wire_adapter is 3.42% slower
- `-3.31%` = wire_adapter has 3.31% lower throughput
- Performance difference is minimal (<5%)

## Expected Performance Summary

| Benchmark | PostgresAdapter | FraiseWireAdapter | Winner |
|-----------|-----------------|-------------------|--------|
| 10K latency | 32ms | 33ms | PostgresAdapter (3% faster) |
| 100K latency | 320ms | 330ms | PostgresAdapter (3% faster) |
| 1M latency | 4.2s | 4.0s | FraiseWireAdapter (5% faster) ⚡ |
| 10K memory | 260 KB | 1.3 KB | FraiseWireAdapter (200x) ⭐ |
| 100K memory | 26 MB | 1.3 KB | FraiseWireAdapter (20,000x) ⭐⭐ |
| 1M memory | 260 MB | 1.3 KB | FraiseWireAdapter (200,000x) ⭐⭐⭐ |
| Pagination | ✅ Excellent | ⚠️  Good | PostgresAdapter |
| WHERE clauses | ✅ Excellent | ✅ Excellent | Tie |

**Summary**:

- **Speed**: Comparable (PostgresAdapter 3-5% faster for small queries, fraiseql-wire catches up or wins on large queries)
- **Memory**: fraiseql-wire wins by **orders of magnitude** (200x to 200,000x improvement)
- **Use Case**: Choose based on workload characteristics

## Troubleshooting

### "DATABASE_URL not set"

```bash
export DATABASE_URL="postgresql:///fraiseql_bench"
```

### "Test data not found in v_benchmark_data"

```bash
psql $DATABASE_URL < benches/fixtures/setup_bench_data.sql
```

### "No database adapters enabled"

```bash
# Enable at least one adapter
cargo bench --bench adapter_comparison --features postgres
# or
cargo bench --bench adapter_comparison --features wire-backend
```

### Benchmarks too slow

```bash
# Reduce sample size (edit benches/adapter_comparison.rs)
group.sample_size(10);  # Default is 100

# Run specific benchmarks only
cargo bench --bench adapter_comparison -- "10k_rows"
```

### Out of memory during 1M row benchmark

```bash
# Skip large benchmarks
cargo bench --bench adapter_comparison -- "10k_rows|100k_rows"

# Or increase system limits
ulimit -v unlimited  # Linux
```

## Adding Custom Benchmarks

### Template

```rust
#[cfg(feature = "postgres")]
fn bench_custom(c: &mut Criterion) {
    let Some(conn_str) = get_connection_string() else {
        return;
    };

    let rt = Runtime::new().unwrap();

    let mut group = c.benchmark_group("custom_bench");

    group.bench_function("my_test", |b| {
        b.to_async(&rt).iter(|| async {
            let adapter = PostgresAdapter::new(&conn_str).await.unwrap();

            // Your custom benchmark code here
            let results = adapter.execute_where_query(...).await.unwrap();

            black_box(results);
        });
    });

    group.finish();
}
```

### Register Benchmark

Add to `criterion_group!` macro at bottom of file:

```rust
criterion_group!(
    postgres_benches,
    bench_postgres_10k_rows,
    bench_custom  // <-- Add here
);
```

## CI/CD Integration

### GitHub Actions Example

```yaml
name: Benchmarks

on:
  push:
    branches: [main]
  pull_request:

jobs:
  benchmark:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:17
        env:
          POSTGRES_DB: fraiseql_bench
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v3
      - uses: actions-rust-lang/setup-rust-toolchain@v1

      - name: Setup test data
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/fraiseql_bench
        run: psql $DATABASE_URL < benches/fixtures/setup_bench_data.sql

      - name: Run benchmarks
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/fraiseql_bench
        run: cargo bench --bench adapter_comparison --features "postgres,wire-backend"

      - name: Archive benchmark results
        uses: actions/upload-artifact@v3
        with:
          name: benchmark-results
          path: target/criterion
```

## References

- [Criterion.rs Guide](https://bheisler.github.io/criterion.rs/book/index.html)
- [tokio-postgres Documentation](https://docs.rs/tokio-postgres/)
- [fraiseql-wire Documentation](https://github.com/fraiseql/fraiseql-wire)
- [PostgreSQL EXPLAIN ANALYZE](https://www.postgresql.org/docs/current/sql-explain.html)

## Contributing

When adding new benchmarks:

1. ✅ Use meaningful benchmark names
2. ✅ Set appropriate `sample_size` for long-running benchmarks
3. ✅ Add `Throughput` for rate-based metrics
4. ✅ Use `black_box()` to prevent compiler optimizations
5. ✅ Document expected results in comments
6. ✅ Add to this README with usage instructions

---

**Happy Benchmarking!** 🚀
