# Copilot Instructions for num-valid

## Project Overview

`num-valid` is a Rust library for robust numerical computation using validated types to prevent floating-point errors like NaN propagation. It provides a generic, layered architecture supporting both native f64 and arbitrary-precision arithmetic (via the `rug` crate).

**Key Concept:** Use validated wrapper types (like `RealNative64StrictFinite`) instead of raw primitives to enforce correctness at the type level.

## Critical Requirements

### Rust Nightly Toolchain
- **Required:** This project uses unstable Rust features (`trait_alias`, `associated_const_equality`, `error_generic_member_access`)
- The toolchain is configured in `rust-toolchain.toml` with `channel = "nightly"`
- Never suggest stable-only solutions - assume nightly features are available

### Feature Flags
- **Default:** Native f64 backend with strict validation
- **`rug`:** Enables arbitrary-precision arithmetic (LGPL-3.0 licensed dependency)
- **`backtrace`:** Enables backtrace capture in errors (disabled by default for performance)

## Architecture: 4-Layer Design

Understanding this layered architecture is essential for making changes.

**📚 For comprehensive details, see [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md)** - a complete 25+ page guide covering:
- Detailed explanation of each layer with examples and rationale
- Step-by-step guides for adding backends and mathematical functions
- Error handling architecture and performance considerations
- Design patterns reference and macro system documentation

**Quick Reference:**

### Layer 1: Raw Trait Contracts (`src/kernels.rs`)
- **Traits:** `RawScalarTrait`, `RawRealTrait`, `RawComplexTrait`
- **Purpose:** Low-level, unchecked operations (e.g., `unchecked_sqrt`, `unchecked_reciprocal`)
- **Contract:** Caller guarantees input validity - no validation performed
- **Key Pattern:** All methods prefixed with `unchecked_*` assume valid inputs
- **Example implementations:** `f64`, `Complex<f64>`, `rug::Float`, `rug::Complex`

### Layer 2: Validation Policies (`src/validation.rs`)
- **Core trait:** `NumKernel` - bundles raw types with validation policies
- **Primary policy:** `StrictFinitePolicy` - ensures values are finite and not subnormal
- **Alternative:** `DebugValidationPolicy<P>` - validates only in debug builds
- **Type-level guarantees:** `GuaranteesFiniteValues` marker enables `Eq` + `Hash` for HashMap keys

### Layer 3: Validated Wrappers (`src/kernels/validated.rs`)
- **Types:** `RealValidated<K>`, `ComplexValidated<K>` (generic over `NumKernel`)
- **Concrete types:** `RealNative64StrictFinite`, `ComplexNative64StrictFinite`, etc.
- **Pattern:** Validate input → call `unchecked_*` method → validate output
- **Copy semantics:** Automatically `Copy` when underlying raw type is `Copy`

### Layer 4: High-Level Traits (`src/lib.rs`)
- **Core trait:** `FpScalar` - universal interface for all scalar types
- **Specialized:** `RealScalar` (for real numbers), `ComplexScalar` (for complex)
- **Mutual exclusion:** Associated type `Kind` (sealed) prevents a type being both real and complex
- **Usage:** Write generic code using `T: RealScalar` bounds

## Common Development Patterns

### Adding Mathematical Functions
1. Define trait in `src/functions/<category>.rs` (e.g., `trigonometric.rs`)
2. Implement for raw types (`f64`, `Complex<f64>`, rug types if needed)
3. Use `#[duplicate_item]` macro to reduce boilerplate across kernels
4. Provide both panicking (`fn sqrt()`) and fallible (`fn try_sqrt()`) variants
5. Define specific error types for the function's domain (e.g., `SqrtRealInputErrors`)

### Macro Usage Pattern
The codebase heavily uses the `duplicate` crate for code generation:
```rust
#[duplicate_item(
    kernel_type;
    [Native64StrictFinite];
    [Native64StrictFiniteInDebug];
)]
impl MyTrait for kernel_type {
    // implementation
}
```
This reduces duplication across validation policies and backends.

### Error Handling Convention
- **Input errors:** Issues with function arguments (e.g., negative value for sqrt)
- **Output errors:** Results that violate validation policy (e.g., operation produces NaN)
- All error types derive from `thiserror::Error`
- Backtraces captured only when `backtrace` feature enabled (see `capture_backtrace()`)

### Testing Strategy
- Tests organized in module-level `mod tests` blocks
- Feature-gated tests for `rug` backend: `#[cfg(feature = "rug")]`
- Generic test functions parameterized by scalar type (e.g., `test_add::<T: FpScalar>()`)
- Run tests with: `cargo test --features=rug`
- For panicking constructors, use `#[should_panic(expected = "...")]` attribute

### API Design Patterns
- **Fallible + Panicking Pairs**: Provide both `try_*()` and panicking variants
  - Example: `try_from_f64()` (returns `Result`) + `from_f64()` (panics on error)
  - Panicking versions useful for constants and tests where validity is guaranteed
  - Document panic conditions clearly in docstrings

### Naming Conventions
- **`kernel_` Prefix Policy**: Reserved ONLY for primitive type operations that directly manipulate underlying type properties
  - **WITH `kernel_` prefix**: Rounding (`kernel_ceil`, `kernel_floor`, `kernel_round`, `kernel_trunc`, `kernel_fract`) and Sign operations (`kernel_copysign`, `kernel_signum`, `kernel_is_sign_positive`, `kernel_is_sign_negative`)
  - **WITHOUT `kernel_` prefix**: Mathematical functions (`sin`, `cos`, `sqrt`, `exp`, `ln`, `clamp`, `classify`, `exp_m1`, `ln_1p`, `hypot`, etc.)
  - **Rationale**: Distinguishes primitive operations from mathematical computations
  - **Method conflict resolution**: When trait methods have same names as f64 inherent methods (e.g., `clamp`, `classify`), use UFCS in doctests: `Clamp::clamp(value, &min, &max)` instead of `value.clamp(&min, &max)`

### Performance Optimization Pattern (rug backend)
- **Use MPFR precomputed constants** when available instead of calculations
  - Example: `MpfrConstant::Pi` (~10x faster than `acos(-1)`)
  - Example: `MpfrConstant::Log2` (~10x faster than `ln(2)`)
  - Available: `Pi`, `Log2`, `Euler`, `Catalan`
  - Not available: `E` (must use `exp(1)`)
- **Cannot use LazyLock caching** inside `impl<const PRECISION>` (Rust limitation)
- Raw functions (`raw_pi`, `raw_ln_2`) should use MPFR constants directly
- Validated wrapper uses generic impl in `src/kernels/validated.rs`

## Build & Test Commands

```bash
# Format code (always run before committing)
cargo fmt

# Run tests for native backend
cargo test

# Run tests with rug (arbitrary-precision)
cargo test --features=rug

# Run benchmarks
cargo bench

# Generate documentation with rug feature
cargo doc --features=rug --open

# Check coverage (requires tarpaulin)
cargo tarpaulin --config tarpaulin.toml
```

## Key Files & Directories

- `src/lib.rs` - Main entry point, high-level traits (`FpScalar`, `RealScalar`, `ComplexScalar`)
- `src/kernels.rs` - Layer 1 raw traits and Layer 2 kernel configuration
- `src/kernels/native64.rs` - Native f64 backend implementation
- `src/kernels/rug.rs` - Arbitrary-precision backend (feature-gated)
- `src/kernels/validated.rs` - Layer 3 validated wrapper implementations
- `src/validation.rs` - Validation policies and error types
- `src/functions/` - Mathematical function traits organized by category
- `benches/performance_benchmark.rs` - Performance comparison benchmarks

## Type System Conventions

### Naming Patterns
- `Real*` - Real number types
- `Complex*` - Complex number types  
- `*Native64*` - Uses f64 backend
- `*Rug*` - Uses arbitrary-precision backend
- `*StrictFinite` - Always validates (debug + release)
- `*StrictFiniteInDebug` - Validates only in debug mode
- `Raw*Trait` - Layer 1 unchecked operations
- `*Validated` - Layer 3 validated wrappers

### Associated Types Pattern
- `RealType` - The real component type (for complex scalars)
- `RawReal` / `RawComplex` - Underlying raw type
- `Kind` - Sealed type for real/complex distinction

## Special Considerations

### Precision in rug Backend
- Rug types use const generic precision: `RealRugStrictFinite<100>` (100-bit precision)
- Always validate f64→rug conversion for exact representability
- Precision mismatches trigger `ErrorsTryFromf64::IncorrectPrecision`

### Bytemuck Integration
- `RealNative64StrictFinite` implements `CheckedBitPattern` + `NoUninit`
- Enables zero-copy conversions from byte arrays
- Invalid bit patterns (NaN, infinity, subnormals) rejected during conversion

### Reference vs. Value Semantics
- Prefer reference-based operations to avoid cloning arbitrary-precision types
- All arithmetic ops support: `T op T`, `T op &T`, `&T op T`, `&T op &T`
- Max/Min use reference-based API (not `Ord` trait) for efficiency

### Compensated Summation
- Default `Sum` implementation uses Neumaier algorithm for accuracy
- Example: `data.iter().sum::<RealNative64StrictFinite>()` uses compensated sum
- See `src/neumaier_compensated_sum.rs`

## Documentation Standards

- All public items require doc comments with examples
- Use intra-doc links: `[`RealScalar`]` instead of plain text
- Include error scenarios in function docs
- Document validation behavior (when checks occur)
- Mark unstable features with clear warnings

## Precision Requirements (rug Backend)

### Choosing Precision Values
- **Precision is bits in significand**, not decimal digits
- Common values: `53` (f64 equivalent), `100`, `200`, `500`, `1000`
- Precision conversion: `decimal_digits ≈ precision * log10(2) ≈ precision * 0.301`
- Example: 100-bit precision ≈ 30 decimal digits

### Type Declaration Pattern
```rust
// Define precision as const generic
type MyReal = RealRugStrictFinite<100>;  // 100-bit precision
type MyComplex = ComplexRugStrictFinite<100>;

// Precision must match across compatible types
let a = MyReal::try_from_f64(1.5)?;  // OK: validates exact representation
let b = MyReal::try_from_f64(0.1)?;  // Error: 0.1 not exactly representable
```

### Precision Validation Rules
- `try_from_f64()` requires **exact representation** at target precision
- Precision mismatches between types trigger compile-time errors
- Operations preserve precision: `RealRugStrictFinite<100> + RealRugStrictFinite<100> = RealRugStrictFinite<100>`
- Use associated type bounds to enforce precision: `where K: NumKernel<RawReal = rug::Float, PRECISION = 100>`

### Performance vs. Precision Trade-offs
- **53 bits:** Native f64 precision, minimal overhead
- **100-200 bits:** Good balance for most scientific computing
- **500+ bits:** High precision but noticeable performance impact
- **1000+ bits:** Use only when mathematically required

## CI/CD and GitLab Pipeline

### Pipeline Structure
The project uses GitLab CI with comprehensive test coverage:
- **4 build configurations:** Debug/Release × with/without rug
- **Test matrix:** All feature combinations tested separately
- **Coverage reporting:** `tarpaulin` with 4 configurations (see `tarpaulin.toml`)
- **Security:** `cargo-audit` runs on schedule
- **Formatting/Linting:** `cargo fmt` and `clippy` checks

### Running Coverage Locally
```bash
# Install tarpaulin
cargo install cargo-tarpaulin

# Run specific configuration from tarpaulin.toml
cargo tarpaulin --config debug_no_rug
cargo tarpaulin --config release_rug

# Generate HTML report
cargo tarpaulin --out Html --output-dir coverage/
```

### CI Requirements
- **Nightly toolchain:** Pipeline uses `rustlang/rust:nightly` Docker image
- **System dependencies:** libgmp-dev, libmpfr-dev, libmpc-dev (for rug)
- **Coverage threshold:** Aim for ≥90% (green badge), minimum 75% (yellow)

### Badge Configuration
Update README badges with your GitLab username:
```markdown
[![Pipeline Status](https://gitlab.com/YOUR_USERNAME/num-valid/badges/master/pipeline.svg)]
[![Coverage](https://gitlab.com/YOUR_USERNAME/num-valid/badges/master/coverage.svg)]
```

Enable coverage parsing in GitLab Settings → CI/CD with regex: `Coverage: (\d+\.?\d*)%`

## Benchmarking and Performance

### Running Benchmarks
```bash
# Run all benchmarks
cargo bench

# Run specific benchmark
cargo bench --bench performance_benchmark

# Generate detailed reports (HTML output in target/criterion/)
cargo bench -- --verbose
```

### Benchmark Organization
- Located in `benches/performance_benchmark.rs`
- Uses `criterion` crate for statistical analysis
- Test vector size: 10,000 elements (adjust `VECTOR_SIZE` constant for different scales)

### Performance Expectations

#### Native64 Backend (f64)
- **Validated types overhead:** ~5-15% compared to raw f64 (Release mode)
- **StrictFinite vs StrictFiniteInDebug:** Identical performance in Release builds
- **Compensated summation:** ~2-3x slower than naive sum, but numerically superior

#### Rug Backend (arbitrary-precision)
- **Precision impact:** Linear scaling with bit width for most operations
- **100-bit precision:** ~10-50x slower than f64 depending on operation
- **Memory usage:** Increases with precision (approximately `precision/8` bytes per value)

### Optimization Checklist
- Use `StrictFiniteInDebug` for performance-critical Release builds
- Prefer `&T` (references) over `T` (values) for rug types to avoid clones
- Use `mul_add_ref()` instead of separate multiply and add
- Consider `fold()` instead of `sum()` when accuracy is less critical than speed

## Contributing and Release Workflow

### Version Management
- **Versioning:** Follows [Semantic Versioning](https://semver.org/) (MAJOR.MINOR.PATCH)
- **Release tool:** Uses `cargo-release` configured in `Release.toml`
- **Protected branch:** Releases only from `master` branch
- **Changelog:** Maintained manually in `CHANGELOG.md` following [Keep a Changelog](https://keepachangelog.com/)

### Release Process
```bash
# 1. Update CHANGELOG.md with new version section
# Move items from [Unreleased] to [X.Y.Z] - YYYY-MM-DD

# 2. Run cargo-release (dry-run first)
cargo release patch --dry-run   # For bug fixes (0.2.4 -> 0.2.5)
cargo release minor --dry-run   # For new features (0.2.4 -> 0.3.0)
cargo release major --dry-run   # For breaking changes (0.2.4 -> 1.0.0)

# 3. Execute release (signs commits and tags)
cargo release patch --execute

# 4. Verify GitLab pipeline passes
# 5. Manually publish to crates.io (publish = false in Release.toml)
cargo publish
```

### Commit and Change Conventions

#### Changelog Categories
- **Added:** New features, traits, functions, or types
- **Changed:** Modifications to existing functionality
- **Fixed:** Bug fixes
- **Removed:** Deprecated/removed features

#### Feature Flag Changes
When adding/modifying features:
1. Update `Cargo.toml` feature flags
2. Add feature-gated tests: `#[cfg(feature = "new_feature")]`
3. Update documentation in both README.md and lib.rs
4. Add examples demonstrating the feature
5. Update CHANGELOG.md under "Added" or "Changed"

#### Breaking Changes
- Increment MAJOR version
- Clearly document migration path in CHANGELOG
- Consider deprecation period for public APIs
- Update all examples and tests

### Pre-Commit Checklist
```bash
# Format code
cargo fmt

# Check compilation for all feature combinations
cargo check
cargo check --features=rug
cargo check --features=backtrace
cargo check --all-features

# Run tests
cargo test
cargo test --features=rug
cargo test --all-features

# Check for common mistakes
cargo clippy -- -D warnings

# Verify documentation builds
cargo doc --features=rug --no-deps
```

### Documentation Updates
- **API changes:** Update doc comments with examples
- **Architecture changes:** Update `.github/copilot-instructions.md`
- **User-facing changes:** Update README.md
- **Internal changes:** Document in code comments and design docs

## When Working With This Codebase

**📖 Essential Reading:** Before making significant changes, read [`docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md) to understand the design philosophy and implementation patterns.

1. **Before adding raw operations:** Ensure they follow the `unchecked_*` naming convention
2. **When adding traits:** Consider using `duplicate_item` macro to reduce boilerplate
3. **For new validation policies:** Implement `GuaranteesFiniteValues` if guaranteeing finite values
4. **Adding rug support:** Always feature-gate with `#[cfg(feature = "rug")]`
5. **Error types:** Use `thiserror` and follow input/output error categorization
6. **Generic bounds:** Use `FpScalar`, `RealScalar`, or `ComplexScalar` for maximum flexibility
7. **Before committing:** Run `cargo fmt ; cargo test --features=rug`
8. **Performance-critical code:** Benchmark before and after with `cargo bench`
9. **Breaking changes:** Update CHANGELOG and consider MAJOR version bump
10. **Documentation changes:** Update README.md, lib.rs docs, and ARCHITECTURE.md as needed
