# TensorLogic Adapters Benchmarks

This directory contains comprehensive benchmarks for the `tensorlogic-adapters` crate, measuring performance across core functionality.

## 📊 Benchmark Suites

### 1. Symbol Table Benchmarks (`symbol_table_benchmarks.rs`)

**Purpose**: Measure core symbol table operations performance.

**Benchmarks**:
- Domain operations (add, lookup, modification)
- Predicate operations (add, lookup, signature matching)
- Variable binding and lookup
- Serialization (JSON/YAML round-trip)
- Compact schema conversion
- Large schema operations (100+ domains, 200+ predicates)

**Usage**:
```bash
cargo bench --bench symbol_table_benchmarks
```

**Key Metrics**:
- Domain lookup: ~50-100ns (O(1) hash map)
- Predicate lookup: ~50-100ns (O(1) hash map)
- JSON serialization: ~5-10µs for small schemas
- Compact conversion: ~2-5µs overhead

### 2. Incremental Validation Benchmarks (`incremental_validation_benchmarks.rs`)

**Purpose**: Demonstrate performance gains from incremental validation.

**Benchmarks**:
- Full validation vs incremental validation
- Change tracking overhead
- Cache hit rate effectiveness
- Batch operation performance
- Large schema validation (100+ components)

**Usage**:
```bash
cargo bench --bench incremental_validation_benchmarks
```

**Key Metrics**:
- Incremental speedup: **10-100x** for large schemas
- Cache hit rate: **90%+** for typical workflows
- Change tracking overhead: <5% of validation time
- Batch validation: Linear scaling with batch size

**Performance Tips**:
- Use batch operations for multiple related changes
- Leverage incremental validation during iterative development
- Clear change tracker periodically to avoid memory growth

### 3. Query Planner Benchmarks (`query_planner_benchmarks.rs`)

**Purpose**: Measure query planning and execution performance.

**Benchmarks**:
- Query by name, arity, signature, domain
- Pattern-based queries
- Complex queries (AND/OR combinations)
- Plan caching effectiveness
- Large schema queries (100+ predicates)

**Usage**:
```bash
cargo bench --bench query_planner_benchmarks
```

**Key Metrics**:
- Simple query (by name): ~200-500ns
- Arity query: ~1-5µs (depends on schema size)
- Pattern query: ~5-20µs (depends on complexity)
- Cache hit: ~50-100ns (near-instant)
- Cache miss: ~1-10µs (depends on query type)

**Performance Tips**:
- Use specific queries (by name) when possible
- Leverage query cache for repeated queries
- Pattern queries are more expensive but flexible
- Consider creating custom indices for frequent queries

### 4. Schema Evolution Benchmarks (`schema_evolution_benchmarks.rs`)

**Purpose**: Measure schema comparison and evolution analysis.

**Benchmarks**:
- Schema diff computation
- Breaking change detection
- Migration plan generation
- Compatibility checking
- Large schema evolution (100+ components)

**Usage**:
```bash
cargo bench --bench schema_evolution_benchmarks
```

**Key Metrics**:
- Small schema diff: ~5-20µs
- Large schema diff: ~100-500µs
- Breaking change detection: ~50-200µs
- Migration plan: ~100-500µs

**Performance Tips**:
- Schema comparison is O(n) in schema size
- Cache evolution reports for repeated comparisons
- Use versioning to minimize diff computation

## 🚀 Running All Benchmarks

Run all benchmark suites:
```bash
cargo bench --benches
```

Run specific benchmark:
```bash
cargo bench --bench symbol_table_benchmarks
```

Run benchmarks matching a pattern:
```bash
cargo bench -- domain_lookup
```

## 📈 Performance Guidelines

### Small Schemas (< 50 components)
- All operations are fast (<100µs)
- Full validation is acceptable
- Query planning overhead is minimal

### Medium Schemas (50-200 components)
- Use incremental validation for iterative changes
- Query planner cache becomes beneficial
- Schema evolution analysis is still fast

### Large Schemas (200+ components)
- **Incremental validation is essential** (10-100x speedup)
- **Query planner cache is critical** (near-instant cached queries)
- Consider schema partitioning for very large schemas
- Batch operations reduce validation overhead

### Performance Optimization Checklist

✅ **Use batch operations** for multiple changes
- `ChangeTracker::begin_batch()` / `end_batch()`
- Reduces validation overhead by 50%+

✅ **Enable query plan caching**
- Enabled by default in `QueryPlanner`
- Clear cache if memory is a concern

✅ **Leverage incremental validation**
- Track changes with `ChangeTracker`
- Use `IncrementalValidator` for repeated validation
- 10-100x faster for large schemas

✅ **Choose appropriate query types**
- By name: Fastest (~200ns)
- By arity: Fast (~1-5µs)
- By pattern: Flexible but slower (~5-20µs)

✅ **Minimize schema evolution analysis**
- Cache evolution reports
- Use versioning to track changes
- Only analyze when schemas actually change

## 🔬 Interpreting Results

### Criterion Output

Criterion provides statistical analysis:
```
domain_lookup           time:   [58.234 ns 58.891 ns 59.612 ns]
                        change: [-2.1234% +0.5678% +3.2345%] (p = 0.65 > 0.05)
                        No change in performance detected.
```

**Key metrics**:
- **time**: Mean execution time with confidence interval
- **change**: Performance change from previous run
- **p-value**: Statistical significance (p < 0.05 = significant)

### Performance Targets

**Excellent** (< 1µs):
- Domain/predicate lookup
- Variable binding lookup
- Cached query execution

**Good** (1-10µs):
- Arity-based queries
- Small schema validation
- Compact schema conversion

**Acceptable** (10-100µs):
- Pattern-based queries
- Medium schema validation
- Schema diff computation

**Needs Optimization** (> 100µs):
- Large schema full validation without incremental
- Complex multi-pattern queries
- Deep schema evolution analysis

## 🛠️ Benchmark Development

### Adding New Benchmarks

1. Create a new file in `benches/`:
```rust
use criterion::{criterion_group, criterion_main, Criterion};
use tensorlogic_adapters::*;

fn my_benchmark(c: &mut Criterion) {
    c.bench_function("my_operation", |b| {
        b.iter(|| {
            // Operation to benchmark
        });
    });
}

criterion_group!(benches, my_benchmark);
criterion_main!(benches);
```

2. Add to `Cargo.toml`:
```toml
[[bench]]
name = "my_benchmarks"
harness = false
```

3. Run:
```bash
cargo bench --bench my_benchmarks
```

### Best Practices

- **Setup outside `iter()`**: Expensive setup should be outside the benchmark loop
- **Use `black_box()`**: Prevent compiler optimizations from eliminating code
- **Consistent inputs**: Use fixed seeds for reproducible results
- **Multiple sizes**: Test with small, medium, and large inputs
- **Statistical significance**: Run enough iterations for stable results

## 📊 Continuous Performance Monitoring

### CI Integration

Add to GitHub Actions:
```yaml
- name: Run benchmarks
  run: cargo bench --benches -- --save-baseline main
```

### Performance Regression Detection

Compare against baseline:
```bash
cargo bench --bench symbol_table_benchmarks -- --baseline main
```

### Generating Reports

Criterion generates HTML reports in `target/criterion/`:
```bash
open target/criterion/report/index.html
```

## 🎯 Performance Goals

### v0.1.0 Targets

- ✅ Domain lookup: < 100ns
- ✅ Predicate lookup: < 100ns
- ✅ Small schema validation: < 10µs
- ✅ Incremental validation: 10x+ speedup
- ✅ Query cache hit: < 100ns
- ✅ Schema diff (small): < 20µs

### Future Optimizations

- [ ] SIMD-accelerated validation
- [ ] Parallel schema diff computation
- [ ] Streaming serialization for large schemas
- [ ] Custom memory allocators
- [ ] Zero-copy deserialization

## 📚 References

- **Criterion.rs**: https://bheisler.github.io/criterion.rs/book/
- **Rust Performance Book**: https://nnethercote.github.io/perf-book/
- **Benchmarking Best Practices**: https://easyperf.net/blog/

---

**Last Updated**: 2025-11-17
**Maintained By**: TensorLogic Team
