# mimirs-embed

![mimirs-embed concept](../docs/assets/mimirs_embed_concept.png)


Multimodal document embedding provider using the LFM2.5-VL-450M model.

## What It Does

- `Lfm2Embedder`: Produces 256-dimensional f64 embeddings from text, images, PDFs, Office documents, audio, video, and archive files via the LFM2.5-VL-450M GGUF model and `llama.cpp`.
- `EmbedError`: Error enum with 10 variants covering I/O, model availability, format support, and extraction failures.
- Falls back to deterministic hash-based embeddings when `llama-embedding` is unavailable.

## Supported Formats

| Category | Extensions |
|----------|-----------|
| Text/code | `.txt`, `.md`, `.rst`, `.log`, `.rs`, `.py`, `.js`, `.ts`, `.go`, `.c`, `.cpp`, `.h`, `.hpp`, `.java`, `.rb`, `.sh`, `.toml`, `.yaml`, `.yml`, `.json`, `.xml`, `.html`, `.htm`, `.css`, `.scss`, `.less`, `.sql`, `.php`, `.swift`, `.kt`, `.scala`, `.r`, `.m`, `.mm`, `.lua`, `.pl`, `.pm`, `.vim`, `.el`, `.lisp`, `.hs`, `.ml`, `.fs`, `.fsx`, `.clj`, `.ex`, `.exs`, `.erl`, `.hrl` |
| Rich text | `.rtf` |
| Word | `.docx`, `.doc` (via antiword) |
| Excel | `.xlsx`, `.xls`, `.csv`, `.tsv` |
| PowerPoint | `.pptx`, `.ppt` (via libreoffice) |
| OpenDocument | `.odt`, `.ods`, `.odp` |
| ePub | `.epub` |
| LaTeX | `.tex`, `.latex`, `.bib` |
| PDF | `.pdf` |
| Images | `.png`, `.jpg`, `.jpeg`, `.webp`, `.bmp`, `.gif`, `.tiff`, `.tif`, `.svg`, `.ico`, `.heic`, `.heif`, `.avif` |
| Audio | `.mp3`, `.wav`, `.flac`, `.ogg`, `.m4a`, `.aac`, `.wma`, `.opus` |
| Video | `.mp4`, `.mkv`, `.avi`, `.webm`, `.mov`, `.wmv`, `.flv`, `.m4v`, `.mpg`, `.mpeg`, `.3gp` |
| Archives | `.zip`, `.tar`, `.gz`, `.bz2`, `.xz`, `.7z`, `.rar` |

## What It Does NOT Do

- No persistence, no reservoir computing, no OTOC, no identity management.
- No contradiction detection, no consolidation.

## Public API

```rust
pub const EMBED_DIM: usize = 256;
pub const MODEL_URL: &str = "https://huggingface.co/LiquidAI/LFM2.5-VL-450M-GGUF/resolve/main/LFM2.5-VL-450M-Q8_0.gguf?download=true";
pub const MODEL_FILENAME: &str = "LFM2.5-VL-450M-Q8_0.gguf";

pub struct Lfm2Embedder {
    /* internal: model_path, cache_dir */
}

impl Lfm2Embedder {
    pub fn new() -> Self;

    /// Embeds a text string. Falls back to hash-based embedding on failure.
    pub fn embed(&self, text: &str) -> Array1<f64>;

    /// Embeds an image file. Falls back to hash-based embedding on failure.
    pub fn embed_image(&self, path: &Path) -> Array1<f64>;

    /// Embeds text and an image jointly. Falls back to hash-based embedding on failure.
    pub fn embed_multimodal(&self, text: &str, image_path: &Path) -> Array1<f64>;

    /// Embeds a PDF document.
    pub fn embed_pdf(&self, path: &Path) -> Array1<f64>;

    /// Embeds a PowerPoint file.
    pub fn embed_presentation(&self, path: &Path) -> Array1<f64>;

    /// Embeds a Word document.
    pub fn embed_document(&self, path: &Path) -> Array1<f64>;

    /// Embeds a spreadsheet.
    pub fn embed_spreadsheet(&self, path: &Path) -> Array1<f64>;

    /// Embeds an OpenDocument file.
    pub fn embed_opendocument(&self, path: &Path) -> Array1<f64>;

    /// Embeds an ePub file.
    pub fn embed_epub(&self, path: &Path) -> Array1<f64>;

    /// Embeds an RTF file.
    pub fn embed_rtf(&self, path: &Path) -> Array1<f64>;

    /// Embeds a LaTeX file.
    pub fn embed_latex(&self, path: &Path) -> Array1<f64>;

    /// Embeds an audio file (metadata extraction, falls back to hash).
    pub fn embed_audio(&self, path: &Path) -> Array1<f64>;

    /// Embeds a video file (keyframe extraction, falls back to hash).
    pub fn embed_video(&self, path: &Path) -> Array1<f64>;

    /// Embeds an archive by extracting and embedding contained text files.
    pub fn embed_archive(&self, path: &Path) -> Array1<f64>;

    /// Auto-detects the file format and embeds accordingly.
    pub fn embed_file(&self, path: &Path) -> Array1<f64>;
}

/// Produces a deterministic 256-dimensional embedding from a byte slice via SHA-256 hashing.
pub fn hash_embedding(data: &[u8]) -> Array1<f64>;

/// Strips RTF formatting commands, returning plain text.
pub fn strip_rtf(raw: &str) -> String;

/// Extracts text content from XML (used for DOCX/PPTX).
pub fn extract_xml_text(xml: &str) -> String;
```

## Error Types

```rust
pub enum EmbedError {
    Io(std::io::Error),
    ModelNotFound(PathBuf),
    LlamaNotFound(String),
    UnsupportedFormat(String),
    ImageError(String),
    PdfError(String),
    ZipError(String),
    ArchiveError(String),
    XmlError(String),
    Fallback,
}
```

## Usage Example

```rust
use mimirs_embed::Lfm2Embedder;
use std::path::Path;

let embedder = Lfm2Embedder::new();

// Embed text
let embedding = embedder.embed("The capital of France is Paris.");
assert_eq!(embedding.len(), 256);

// Embed a file (auto-detects format)
let file_embedding = embedder.embed_file(Path::new("document.pdf"));
```

## Dependencies

- `mimirs-core`: Foundation types
- `ndarray`: N-dimensional arrays
- `image`: Image decoding and resizing
- `sha2`: SHA-256 hashing for fallback embeddings
- `zip`, `tar`: Archive extraction
- `pdf-extract`: PDF text extraction
- `symphonia`: Audio metadata extraction
- `resvg`: SVG rendering
- `thiserror`: Error derive macro
- `dirs`: Cache directory resolution
- `which`: Binary discovery
- `tempfile` (dev): Temporary files for tests

## Known Limitations

- Produces 256-dimensional embeddings, which are incompatible with the default `Urdarbrunnr` store dimension of 384.
- Requires `llama-embedding` in PATH for non-fallback embeddings. Without it, all embeddings are deterministic hash-based.
- Video embedding requires `ffmpeg` for keyframe extraction.
- Legacy `.doc` and `.ppt` files require `antiword` and `libreoffice` respectively.
- `path.to_str().unwrap()` calls will panic on non-Unicode paths.
