# OCR — Pure Rust Optical Character Recognition

A from-scratch OCR system built in Rust, informed by decades of OCR research and modern deep-learning pipelines. The goal is a clean, extensible, pure-Rust toolchain that can rival Tesseract on CPU and approach EasyOCR/PaddleOCR accuracy.

## Current State

**Compiles and passes 357+ tests.** The full pipeline works end-to-end for printed text on clean images, with a modular architecture supporting both classical and learned recognition:

- **Image preprocessing**: deskew, binarization (Otsu, Sauvola), noise reduction, contrast enhancement, auto-rotate (0°/90°/180°/270°)
- **Layout analysis**: Union-Find CCL, column/line detection, reading-order resolution, table detection, form field extraction
- **Text detection**: `TextDetector` trait with CCL and lightweight CNN implementations
- **Recognition**: Pattern-matching engine (trainable from synthetic fonts) + CRNN (CNN + BiLSTM + CTC) selectable via `--engine`
- **Multi-language**: Unicode script detection (Latin, CJK, Arabic, Cyrillic, Greek, Hebrew, Thai, Devanagari), dictionaries for 25+ languages, per-script CRNN vocabularies
- **Post-processing**: Dictionary-based spell correction, document structure classification (headings, lists, paragraphs), hierarchical Markdown/JSON output
- **Output formats**: text, JSON, hOCR, TSV, ALTO XML, box files, **searchable PDF** (invisible text overlay), **Markdown**, **structured JSON**
- **CLI**: `extract`, `batch`, `layout`, `list-languages`, `check`, `info`, `validate`
- **Web API** (feature `web-api`): HTTP server with multipart upload
- **PDF input** (feature `pdf`)

## What We Learned from Popular OCR Offerings

| Engine | Detection | Recognition | Languages | Strengths |
|--------|-----------|-------------|-----------|-----------|
| **Tesseract** | Connected components + LSTM line finder | Conv + BiLSTM + CTC (VGSL) | 100+ | Mature, tiny binary, great for scanned docs |
| **EasyOCR** | CRAFT (CNN) | CRNN (ResNet + BiLSTM + CTC) | 80+ | Simple PyTorch API, good scene text |
| **PaddleOCR** | DB (differentiable binarization) | SVTR / CRNN | 80+ | Best speed/accuracy trade-off, ultra-light models |
| **TrOCR** | — | Transformer encoder-decoder | 100+ | Excellent on printed text, no detection stage |
| **dots.ocr / VLMs** | End-to-end vision-language | 3B-param multimodal | 100+ | Structured output (tables, headings) but needs GPU |

### Key Architectural Lessons

1. **Two-stage pipeline** (detection → recognition) dominates for documents. End-to-end VLMs are promising but overkill for pure OCR.
2. **CTC decoders** remove the need for per-character segmentation in sequence models.
3. **Synthetic training data** (rendered fonts + realistic distortions) lets you bootstrap accuracy before collecting real data.
4. **Image preprocessing matters**: deskew, binarization, and contrast enhancement can improve accuracy 10–20%.
5. **Post-processing dictionaries / language models** close the gap between raw recognition and human-level accuracy.
6. **CPU deployment** still matters: Tesseract remains ubiquitous because it runs everywhere without GPU dependencies.

## Roadmap

### Phase 0 — Baseline (DONE)
- [x] End-to-end pipeline: image → preprocess → layout → recognize → output
- [x] Pattern-matching recognition engine for Latin glyphs
- [x] Layout analysis with CCL, column detection, reading order
- [x] Dictionary-based post-correction
- [x] CLI with extract, batch, layout, list-languages
- [x] 357+ tests passing

### Phase 1 — Synthetic Training Infrastructure (DONE)
- [x] Synthetic text-image generator with TTF font rendering and bitmap fallback
- [x] Distortion pipeline (rotation, blur, noise, contrast, shear)
- [x] CER/WER benchmark harness with clean/mild/heavy test sets
- [x] Train pattern-matching templates from synthetic renders (`TemplateTrainer`)

### Phase 2 — Robust Text Detection (DONE)
- [x] `TextDetector` trait with `CclDetector` and `CnnDetector`
- [x] Lightweight 3-layer detection CNN in pure Rust (ndarray)
- [x] Synthetic document generator + IoU-based evaluation harness
- [x] Auto-rotate 0°/90°/180°/270° via projection-variance orientation detection

### Phase 3 — Learned Recognition (CRNN) (DONE)
- [x] CNN feature extractor (5 conv layers + maxpool)
- [x] 2-layer bidirectional LSTM (64 hidden)
- [x] CTC decoder (greedy + beam search)
- [x] CTC loss with forward-backward algorithm
- [x] `CrnnTrainer` with synthetic data + checkpoint saving
- [x] Wired into `OcrEngine` via `--engine lstm`
- [x] Model size < 5MB (~2.4MB default config)

### Phase 4 — Multi-Language Scale (DONE)
- [x] Unicode script detection and routing (8 scripts)
- [x] Dictionaries for 25+ languages
- [x] Multi-script synthetic data generation (`ScriptLineGenerator`)
- [x] Per-script CRNN models (`ScriptModelRegistry`)

### Phase 5 — Advanced Layout & Structure (DONE)
- [x] Table detection with grid line scan and cell reconstruction
- [x] Row/column span inference via pixel-density checks
- [x] Form field extraction (checkboxes, key-value pairs, underline fields)
- [x] Document structure classification (headings, paragraphs, lists, captions)
- [x] Hierarchical Markdown and structured JSON output
- [x] Searchable PDF generation with invisible text overlay

### Next Steps (requires training execution)
- [ ] Train CRNN to CER < 5% on clean synthetic test
- [ ] Train CRNN to CER < 15% on distorted synthetic test
- [ ] Evaluate per-language accuracy on synthetic benchmarks

## Installation

```bash
# Default build (pattern matching engine)
cargo build --release

# With all features
cargo build --release --all-features

# Run tests
cargo test
```

## Usage

```bash
# Recognize text from an image
ocr extract document.png

# JSON output with bounding boxes and confidence
ocr extract document.png -f json

# Markdown output with document structure
ocr extract document.png -f markdown

# Structured JSON with headings, paragraphs, lists
ocr extract document.png -f structured-json

# Enable preprocessing pipeline
ocr extract document.png --preprocess

# Use CRNN engine
ocr extract document.png --engine lstm

# Batch process a directory
ocr batch -i ./images -o ./results --lang en

# List supported languages
ocr list-languages

# Analyze image layout
ocr layout document.png -o layout.json
```

### Library

```rust
use ocr::api::Ocr;
use ocr::core::config::OcrConfig;

let config = OcrConfig::default();
let ocr = Ocr::with_config(config)?;
ocr.initialize().await?;

let result = ocr.recognize_text_from_file("document.png").await?;
println!("{}", result.text);
```

## Project Structure

```
src/
├── api/          High-level OCR API (Ocr, TextProcessor)
├── cli/          CLI argument parsing with clap
├── core/         Core OCR engine (config, engine, geometry, layout, output, text)
├── image/        Image preprocessing pipeline (binarization, enhancement, quality)
├── lang/         Language support (CJK, detection, dictionary, N-gram, unicode)
├── layout/       Layout analysis (column/line detection, text ordering, CCL, detection CNN, table/form extractors, classifier)
├── recognition/  Recognition models (pattern matching, CRNN, CTC decoder, LSTM)
├── synthetic/    Synthetic data generation (fonts, distortion, multi-script, document layouts, template training)
├── training/     Model training pipeline (data, losses, metrics, optimizers, CRNN trainer)
└── utils/        Shared utilities (async, error, hash, math, SIMD, time)
```

## License

Apache-2.0
