# llm-extract

Extract structured data from LLM responses in Rust.

**`&str` in, `T` out.** Handles the messy reality of LLM output: markdown fences, prose, `<think>` blocks, trailing commas, single quotes, Python literals, comments, unclosed brackets, field name typos, and enum variant typos.

## Architecture

```
Input text
  -> strip <think> blocks
  -> strip markdown fences + prose
  -> bracket-match JSON entities
  -> jsonrepair (syntax repair)
  -> fuzzy repair (field/variant typos)
  -> serde deserialize -> T
```

**Layer 1** (`extract_json`): `&str -> serde_json::Value` -- no schema needed.

**Layer 2** (`extract`): `&str -> T` -- adds fuzzy repair via `#[derive(Extract)]`.

## Quick Start

```rust
use llm_extract::{extract, extract_json, Extract};
use serde::Deserialize;

// Layer 1: extract as Value
let text = r#"Here's the result: ```json
{"status": "ok", "count": 42}
```"#;
let value = extract_json(text).unwrap();
assert_eq!(value["status"], "ok");

// Layer 2: extract as typed struct with fuzzy repair
#[derive(Deserialize, Extract)]
struct ApiResult {
    status: String,
    count: u32,
}

let result: ApiResult = extract(text).unwrap();
assert_eq!(result.status, "ok");
```

## What It Handles

### Syntax Repair (via [jsonrepair](https://crates.io/crates/jsonrepair))

| Problem | Example |
|---|---|
| Markdown fences | `` ```json\n{...}\n``` `` |
| Prose around JSON | `Sure! {"a": 1} Hope that helps!` |
| `<think>` blocks | `<think>reasoning...</think>{"a": 1}` |
| Trailing commas | `{"a": 1,}` |
| Single quotes | `{'a': 1}` |
| Unquoted keys | `{a: 1}` |
| Python literals | `True`, `False`, `None` |
| Comments | `//`, `/* */`, `#` |
| Unclosed brackets | `{"a": 1` |
| Unclosed strings | `{"a": "test` |

### Semantic Repair (via [fuzzy-parser](https://crates.io/crates/fuzzy-parser))

| Problem | Example |
|---|---|
| Field name typo | `"taget"` -> `"target"` |
| Enum variant typo | `"AddDeriv"` -> `"AddDerive"` |
| Nested struct typos | Recursive repair with `#[extract(nested)]` |

## API

### Core Functions

```rust
// Layer 1: Value extraction
extract_json(text) -> Result<Value, ExtractError>
extract_json_all(text) -> Result<Vec<Value>, ExtractError>

// Layer 2: Typed extraction with repair (requires T: Extract)
extract::<T>(text) -> Result<T, ExtractError>
extract_all::<T>(text) -> Result<Vec<T>, ExtractError>
extract_verbose::<T>(text) -> Result<Extracted<T>, ExtractError>

// Strict: no repair (requires T: DeserializeOwned)
extract_strict::<T>(text) -> Result<T, ExtractError>
```

### `extract_verbose` -- Corrections Metadata

```rust
let result = extract_verbose::<Task>(r#"{"taget": "main.rs", "priority": 1}"#).unwrap();
assert_eq!(result.value.target, "main.rs");
assert!(!result.corrections.is_empty()); // ["taget" -> "target"]
```

### Configurable Pipeline

```rust
use llm_extract::{Extractor, RepairMode, Algorithm};

let extractor = Extractor::builder()
    .max_input_len(512 * 1024)
    .repair(RepairMode::Fuzzy {
        min_similarity: 0.8,
        algorithm: Algorithm::JaroWinkler,
    })
    .build();

let result: MyStruct = extractor.extract(text)?;
```

### Individual Steps

```rust
use llm_extract::{strip_fences, strip_think_blocks, extract_json_entity};

let clean = strip_think_blocks(text);    // remove <think>...</think>
let inner = strip_fences(&clean);        // remove ```json ... ```
let entity = extract_json_entity(inner); // find first {..} or [..]
```

## `derive(Extract)`

```rust
#[derive(Deserialize, Extract)]
struct Action {
    #[serde(rename = "type")]
    action_type: String,
    target: String,
    #[extract(nested = true)]  // enable recursive repair
    options: Options,
}

#[derive(Deserialize, Extract)]
struct Options {
    description: String,
    value: u32,
}

#[derive(Deserialize, Extract)]
#[extract(tag = "type")]
enum ActionKind {
    AddDerive,
    RemoveField,
    RenameVariable,
}
```

### Attributes

**Struct/Enum level:**

| Attribute | Description | Default |
|---|---|---|
| `#[extract(tag = "field")]` | Tagged enum discriminator field | `"type"` |
| `#[extract(similarity = 0.8)]` | Repair similarity threshold | `0.7` |

**Field level:**

| Attribute | Description | Default |
|---|---|---|
| `#[extract(repair = false)]` | Disable repair for this field | `true` |
| `#[extract(nested = true)]` | Enable recursive repair for child struct | `false` |
| `#[extract(similarity = 0.9)]` | Field-level similarity threshold | struct-level |
| `#[extract(aliases = ["colour", "clr"])]` | Additional accepted field names | `[]` |

## Error Handling

```rust
match extract::<MyStruct>(text) {
    Ok(value) => { /* success */ }
    Err(ExtractError::NotFound) => { /* no JSON in input */ }
    Err(ExtractError::Parse { extracted, source }) => { /* JSON found but wrong shape */ }
    Err(ExtractError::RepairFailed { repaired, corrections, source }) => {
        // repair was attempted but deserialization still failed
        // `corrections` shows what was fixed before failure
    }
    Err(ExtractError::InputTooLarge { len, max }) => { /* input exceeds limit */ }
}
```

## Dependencies

- **serde** / **serde_json** -- serialization
- **jsonrepair** -- syntax repair (single quotes, Python literals, comments, etc.)
- **fuzzy-parser** -- semantic repair (field/variant name fuzzy matching)
- **thiserror** -- error types
- **llm-extract-derive** -- `#[derive(Extract)]` proc-macro

## License

MIT OR Apache-2.0
