# mimirs-encoding

![mimirs-encoding concept](../docs/assets/mimirs_encoding_concept.png)


Semantic vectorization using transformer-based models and manifold alignment.

## What It Does

- `SemanticEncoder`: Loads `all-MiniLM-L6-v2` from HuggingFace via the `candle` crate. Produces 384-dimensional f64 embeddings from text. Supports optional Procrustes alignment.
- `ProcrustesAligner`: Aligns embedding spaces across sessions using SVD-based orthogonal Procrustes.
- `SleipnirEngine` (feature-gated `sleipnir`): Continuous-time dynamics encoder using LFM2.5-VL-450M. Produces 256-dimensional embeddings.

## What It Does NOT Do

- No persistence, no reservoir computing, no OTOC.
- No multimodal document processing (use `mimirs-embed` for that).

## Public API

```rust
pub struct SemanticEncoder {
    /* internal: BertModel, Tokenizer, Device */
}

impl SemanticEncoder {
    /// Downloads and loads all-MiniLM-L6-v2 from HuggingFace.
    /// Requires network access on first call.
    pub fn load_default() -> Result<Self, MimirError>;

    /// Attaches a Procrustes rotation matrix for alignment.
    pub fn with_alignment(self, rotation: Tensor) -> Self;

    /// Encodes text into a 384-dimensional f64 vector.
    /// Returns Err on tokenization failure or model forward pass failure.
    pub fn encode(&self, text: &str) -> Result<Vec<f64>, MimirError>;
}

pub type SharedEncoder = Arc<SemanticEncoder>;

pub struct ProcrustesAligner {
    /* internal: rotation matrix */
}

impl ProcrustesAligner {
    pub fn new(rotation: Tensor) -> Self;
    pub fn align(&self, embeddings: &Tensor) -> Result<Tensor, MimirError>;
}

pub struct SleipnirEngine {
    /* internal: LFM2.5-VL model */
}

impl SleipnirEngine {
    /// Loads the model from a GGUF file path.
    pub fn load(path: &Path, device: Device) -> Result<Self, MimirError>;

    /// Encodes text with continuous-time dynamics.
    pub fn encode_with_dynamics(&self, text: &str, t: f64) -> Result<DensityMemory, MimirError>;

    /// Predicts complexity for reservoir epsilon tuning.
    pub fn predict_complexity(&self, text: &str) -> f32;
}
```

## Feature Flags

| Flag | Effect |
|------|--------|
| `sleipnir` | Enables `SleipnirEngine` (LFM2.5-VL continuous-time dynamics) |

## Usage Example

```rust
use mimirs_encoding::SemanticEncoder;

// Downloads all-MiniLM-L6-v2 from HuggingFace on first call
let encoder = SemanticEncoder::load_default().expect("model load failed");

// Encode text
let vector = encoder.encode("The capital of France is Paris.").expect("encoding failed");
assert_eq!(vector.len(), 384);
```

## Dependencies

- `candle-core`, `candle-nn`, `candle-transformers`: ML inference
- `tokenizers`: BPE tokenization
- `hf_hub`: HuggingFace model download
- `mimirs-core`: Foundation types, `MimirError`
- `nalgebra`: Linear algebra
- `serde`, `serde_json`: Config parsing
- `thiserror`: Error derive macro

## Known Limitations

- `load_default()` requires network access to HuggingFace. Cached after first download.
- The `unsafe` block in `load_default()` uses `VarBuilder::from_mmaped_safetensors()` which memory-maps the model file. This is safe when the file is not modified while mapped.
- SVD in `procrustes_align()` (in `src/alignment.rs`) will panic with "SVD U failed" if the input matrix is rank-deficient or contains NaN/Inf.
