# Sutra AFM (Apple Foundation Models for Rust)

`sutra_afm_lib` is a safe, high-performance, and idiomatic Rust library for Apple's on-device foundation models (Apple Intelligence).

Leverage the ~3B parameter large language model built into macOS 15.1+ (up to macOS 26+) and iOS 18.1+ directly from your Rust applications with zero model weight management and native `async/await` support.

## Project Structure

This repository is organized as a Cargo Workspace:
- **`sutra_afm_lib`**: The core library crate. Lightweight and intended for use in any Rust application.
- **`sutra-ai`**: A powerful CLI and TUI application for interacting with the on-device model.

## Features

- **On-Device Inference:** Run LLM prompts locally on Apple Silicon (M1+).
- **Idiomatic Async:** Built-in support for `tokio` and `futures-util` Streams.
- **Stateful Sessions:** Maintain multi-turn conversations with context memory.
- **Personas & Instructions:** Define system-level instructions to control model behavior.
- **Structured Generation (JSON):** Automatically deserialize model output into Rust `structs`.
- **Autonomous Agents:** Register Rust closures as tools and let the `Agent` autonomously execute them.
- **MCP Integration:** Native support for the **Model Context Protocol (MCP)** for orchestrating local/remote tool servers.
- **Local RAG & Embeddings:** Generate 512-dimensional vector embeddings natively using Apple's NLP framework.
- **Privacy First:** Data never leaves the device.

## Requirements

- **Operating System:** macOS 15.1 or newer (optimized for macOS 26).
- **Hardware:** Apple Silicon (M1, M2, M3, M4, etc.).
- **Software:** Xcode 16.1+ (macOS SDK 26.0+).

## Usage (Library)

Add to your `Cargo.toml`:
```toml
[dependencies]
sutra_afm_lib = "0.6.0"
```

### Simple Async Response
```rust
use sutra_afm_lib::FoundationModel;

#[tokio::main]
async fn main() {
    let response = FoundationModel::respond("Explain Rust lifetimes.").await.unwrap();
    println!("{}", response);
}
```

## CLI Tool: `sutra-ai`
The CLI is maintained as a separate crate within the workspace to keep the library dependencies minimal.
To run the CLI from source:
```bash
cargo run -p sutra-ai -- chat
```

## License

MIT License.
