# Using the New Bias Evaluation Features

This guide demonstrates how to use all the new bias evaluation improvements.

## Quick Start

```rust
use anno::eval::bias_config::BiasDatasetConfig;
use anno::eval::demographic_bias::{DemographicBiasEvaluator, create_diverse_name_dataset};

// Create config with all features enabled
let config = BiasDatasetConfig::default()
    .with_frequency_weighting()
    .with_validation()
    .with_detailed(true);

// Create evaluator with config
let evaluator = DemographicBiasEvaluator::with_config(true, config);
let names = create_diverse_name_dataset();
let results = evaluator.evaluate_ner(&model, &names);

// Access all new features
println!("Overall: {:.1}%", results.overall_recognition_rate * 100.0);
println!("Parity gap: {:.1}%", results.ethnicity_parity_gap * 100.0);

// Statistical results
if let Some(stat) = &results.statistical {
    println!("{}", stat.format_with_ci());
}

// Frequency-weighted results
if let Some(freq) = &results.frequency_weighted {
    println!("Weighted: {:.1}%", freq.weighted_rate * 100.0);
}

// Distribution validation
if let Some(validation) = &results.distribution_validation {
    println!("Valid: {}", validation.is_valid);
}

// Extended intersectional analysis
for (key, rate) in &results.extended_intersectional {
    println!("{}: {:.1}%", key, rate * 100.0);
}
```

## Expanded Datasets

All datasets have been significantly expanded:

- **Gender Bias**: 30 → 150 examples (5x)
- **Demographic Bias**: ~100 → ~500+ names (5x)
- **Temporal Bias**: ~70 → ~200+ names (~3x)
- **Length Bias**: ~30 → ~100+ examples (~3x)

## Real-World Contexts

Instead of simple templates like "X attended the conference.", the new system uses 20 diverse realistic sentence contexts:

- "{} was interviewed by the news team."
- "The award was presented to {} at the ceremony."
- "{} published a groundbreaking research paper."
- And 17 more diverse contexts

Contexts are selected deterministically based on name hash for reproducibility.

## Statistical Reporting

All evaluations now support statistical reporting with confidence intervals:

```rust
if let Some(stat) = &results.statistical {
    println!("Mean: {:.3}", stat.mean);
    println!("95% CI: [{:.3}, {:.3}]", stat.ci_95.0, stat.ci_95.1);
    println!("Std dev: {:.3}", stat.std_dev);
    if let Some(effect) = stat.effect_size {
        println!("Effect size: {:.3}", effect);
    }
}
```

## Frequency-Weighted Evaluation

Evaluate with frequency weighting to account for real-world name distributions:

```rust
let config = BiasDatasetConfig::default()
    .with_frequency_weighting();

let evaluator = DemographicBiasEvaluator::with_config(true, config);
let results = evaluator.evaluate_ner(&model, &names);

if let Some(freq) = &results.frequency_weighted {
    println!("Unweighted: {:.1}%", freq.unweighted_rate * 100.0);
    println!("Weighted: {:.1}%", freq.weighted_rate * 100.0);
    println!("Difference: {:.1}%", 
             (freq.weighted_rate - freq.unweighted_rate) * 100.0);
}
```

## Distribution Validation

Validate your dataset against reference distributions (e.g., US Census):

```rust
let config = BiasDatasetConfig::default()
    .with_validation();

let evaluator = DemographicBiasEvaluator::with_config(true, config);
let results = evaluator.evaluate_ner(&model, &names);

if let Some(validation) = &results.distribution_validation {
    println!("Valid: {}", validation.is_valid);
    println!("Max deviation: {:.1}%", validation.max_deviation * 100.0);
    
    if !validation.is_valid {
        println!("Category deviations:");
        for (cat, &dev) in &validation.category_deviations {
            println!("  {}: {:.1}%", cat, dev * 100.0);
        }
    }
}
```

## Extended Intersectional Analysis

Analyze bias across multiple dimensions simultaneously:

```rust
// Results include ethnicity × gender × frequency analysis
for (key, &rate) in &results.extended_intersectional {
    println!("{}: {:.1}%", key, rate * 100.0);
}
```

## Complete Example

See `examples/comprehensive_bias_eval.rs` for a complete example using all features.

Run it with:
```bash
cargo run --example comprehensive_bias_eval --features eval-bias
```

## Configuration Options

```rust
let config = BiasDatasetConfig::default()
    .with_frequency_weighting()      // Enable frequency weighting
    .with_validation()                // Enable distribution validation
    .with_min_samples(30)             // Minimum samples per category
    .with_seeds(vec![42, 123, 456])  // Multiple evaluation seeds
    .with_confidence_level(0.95)     // Confidence level for CI
    .with_detailed(true);             // Include detailed results
```

## Examples

- `examples/benchmark.rs` - Updated to use new features
- `examples/comprehensive_bias_eval.rs` - Complete demonstration
- `examples/bias_dataset_sizes.rs` - Show dataset sizes

