# iqdb-filter v0.2.0 — The Canonical Filter Evaluator

**The semantics land.** v0.2.0 turns the scaffold into a working crate: one place that decides what an `iqdb_types::Filter` means, with validation-on-construction and an infallible per-row boolean evaluator that every metadata-aware index can delegate to. Extracting the evaluator pins a single set of rules across every consumer — the `Neq(absent)` / `Not(Eq(absent))` subtlety can no longer drift between two index implementations and produce query-result bugs nobody can attribute.

## What is iqdb-filter?

The metadata-filtering layer of the iQDB vector database. Given a `Filter` and a record's `Metadata`, it returns a boolean: does this record match? The crate owns the filter semantics so every index above it — `iqdb-flat`, and later `iqdb-hnsw` / `iqdb-ivf` — shares one verified implementation rather than re-deriving the edge cases.

## What's new in 0.2.0

### `FilterEvaluator` — validate once, evaluate many

Construction is the validation gate. `FilterEvaluator::new` walks the filter a single time to enforce the depth and `In`-cardinality caps; after it returns `Ok`, `evaluate` is infallible and runs per-row inside the search loop with nothing to re-check.

```rust
use iqdb_filter::FilterEvaluator;
use iqdb_types::{Filter, Metadata, Value};

let evaluator = FilterEvaluator::new(Filter::and(vec![
    Filter::eq("published", Value::Bool(true)),
    Filter::gt("year", Value::Int(2000)),
]))
.expect("valid filter");

let meta: Metadata = [
    ("published".to_string(), Value::Bool(true)),
    ("year".to_string(), Value::Int(2026)),
]
.into_iter()
.collect();

assert!(evaluator.evaluate(Some(&meta)));
assert!(!evaluator.evaluate(None)); // no metadata -> every leaf is false
```

### Closed-world null / absent-field semantics

Every leaf comparison over an absent field is `false`; type mismatches are `false`; ordered comparisons involving `NaN` are `false`; and `Not` over a `false` leaf is `true`. The upshot is that `Neq(absent)` (`false`) and `Not(Eq(absent))` (`true`) are deliberately **not** interchangeable — the latter is the idiom for "records that lack this field, or carry it with a non-matching value".

```rust
use iqdb_filter::FilterEvaluator;
use iqdb_types::{Filter, Value};

let not_eq =
    FilterEvaluator::new(Filter::not(Filter::eq("author", Value::String("ada".into()))))
        .expect("valid filter");

// A record with no `author` is selected by `Not(Eq)`, but not by `Neq`.
assert!(not_eq.evaluate(None));
```

### DoS hardening built in

The validation walk is **iterative** — an explicit work-list, not recursion — so `new` cannot itself stack-overflow on a pathological input. After construction, every filter is bounded by `MAX_FILTER_DEPTH` (`64`), so the recursive `evaluate` hot path runs with a fixed stack budget; and `MAX_IN_VALUES` (`1024`) caps the `O(n)` membership predicate so an attacker-supplied million-value `In` can't turn a search into a denial of service. Both caps are `pub const` so higher layers can pre-check.

```rust
use iqdb_filter::{FilterEvaluator, MAX_IN_VALUES};
use iqdb_types::{Filter, IqdbError, Value};

let huge = vec![Value::Int(0); MAX_IN_VALUES + 1];
let err = FilterEvaluator::new(Filter::is_in("tag", huge)).unwrap_err();
assert_eq!(err, IqdbError::InvalidFilter);
```

### `FilterStrategy` vocabulary

A `#[non_exhaustive]` enum naming the shapes an index can apply a filter in — `PreFilter`, `PostFilter`, `InFilter`, `Auto`. This release ships the **vocabulary only**: there is no selector yet, and every consumer pre-filters through `FilterEvaluator`. The enum exists so a future `MetadataIndex`-driven index can adopt strategy selection without a breaking change.

### `VERSION`

`iqdb_filter::VERSION` exposes the crate's compile-time `CARGO_PKG_VERSION` for diagnostics and version-skew checks across the family.

## Performance

The per-row evaluation path is allocation-free. On the reference machine, a representative compound predicate — `published AND (year > 2000 OR genre IN {…}) AND NOT archived` — measures:

| Path | Time |
|------|------|
| `evaluate` (match, with metadata) | **~18.8 ns** |
| `evaluate` (no metadata) | ~3.0 ns |
| `new` (one-time validation) | ~370 ns |

Numbers are from the `criterion` bench (`cargo bench`); validation is a one-time cost, evaluation is the path that runs per candidate.

## What's deferred (and why)

The crate's eventual headline — automatic pre/post/in **strategy selection** — is intentionally **not** in this release. So are the optional per-field inverted `MetadataIndex` and selectivity estimation. Each was considered and dropped because it has no real consumer yet: the only metadata-aware index today (`iqdb-flat`) pre-filters with a row scan, and a selectivity estimator with no real workload would bake in the wrong thresholds. They land when the first approximate index (`iqdb-hnsw` / `iqdb-ivf`) starts honouring filters and the cost model actually changes. See `dev/ROADMAP.md` for the full rationale — document, defer, do not stub.

## Breaking changes

**Pre-1.0 API churn.** Relative to the v0.1.x scaffold (no domain logic), everything here is an addition. The crate now depends on `iqdb-types` 1.0 for the shared `Filter` / `Metadata` / `Value` / `IqdbError` / `Result` vocabulary.

## Verification

The semantics are covered by unit tests per module, the `tests/conformance.rs` integration suite (closed-world null/absent rules, depth cap, `In` cap), and `tests/properties.rs` property tests (determinism, `Not` negation, double-negation, `And`/`Or` as conjunction/disjunction, De Morgan, and the closed-world rule over randomly generated filters). The same gates run across the CI matrix (Linux, macOS, Windows) on stable and the 1.87 MSRV:

```bash
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
```

MSRV: Rust 1.87.

## What's next

- **Strategy selection.** When the first approximate index honours filters, `FilterStrategy::{PreFilter, PostFilter, InFilter}` gains a selector driven by a real selectivity estimate, plus the optional inverted `MetadataIndex` it needs.

## Installation

```toml
[dependencies]
iqdb-filter = "0.2"
```

## Documentation

- [README](https://github.com/jamesgober/iqdb-filter/blob/main/README.md)
- [API reference](https://github.com/jamesgober/iqdb-filter/blob/main/docs/API.md)
- [ROADMAP](https://github.com/jamesgober/iqdb-filter/blob/main/dev/ROADMAP.md)
- [Standards (REPS)](https://github.com/jamesgober/iqdb-filter/blob/main/REPS.md)
- [CHANGELOG](https://github.com/jamesgober/iqdb-filter/blob/main/CHANGELOG.md)

---

**Full diff:** [`v0.1.0...v0.2.0`](https://github.com/jamesgober/iqdb-filter/compare/v0.1.0...v0.2.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/iqdb-filter/blob/main/CHANGELOG.md).
