# Aurora Semantic - Developer Integration Guide

This guide covers integrating Aurora Semantic into your IDE or application.

## Table of Contents

1. [Installation](#installation)
2. [Basic Integration](#basic-integration)
3. [Using ONNX Models](#using-onnx-models)
4. [Custom Embedders](#custom-embedders)
5. [Search API](#search-api)
6. [Progress Reporting](#progress-reporting)
7. [Workspace Management](#workspace-management)
8. [Best Practices](#best-practices)
9. [Troubleshooting](#troubleshooting)

## Installation

Add to your `Cargo.toml`:

```toml
[dependencies]
aurora-semantic = "0.1"
tokio = { version = "1", features = ["full"] }
```

## Basic Integration

### Minimal Example

```rust
use aurora_semantic::{Engine, EngineConfig, WorkspaceConfig};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> aurora_semantic::Result<()> {
    // Create engine with default config
    let engine = Engine::new(EngineConfig::default())?;

    // Index current directory
    let ws_config = WorkspaceConfig::new(PathBuf::from("."));
    let workspace_id = engine.index_workspace(ws_config, None).await?;

    // Search
    let results = engine.search_text(&workspace_id, "main function")?;

    println!("Found {} results", results.len());
    Ok(())
}
```

### IDE Integration Pattern

```rust
use aurora_semantic::{Engine, EngineConfig, WorkspaceConfig, ModelConfig};
use std::path::PathBuf;
use std::sync::Arc;

pub struct CodeSearchService {
    engine: Arc<Engine>,
    workspace_id: Option<aurora_semantic::WorkspaceId>,
}

impl CodeSearchService {
    /// Initialize with optional ONNX model
    pub fn new(
        index_dir: PathBuf,
        model_dir: Option<PathBuf>,
    ) -> aurora_semantic::Result<Self> {
        let config = EngineConfig::new(index_dir);

        let engine = if let Some(model_path) = model_dir {
            let embedder = ModelConfig::from_directory(model_path).load()?;
            Engine::with_embedder(config, embedder)?
        } else {
            Engine::new(config)?
        };

        Ok(Self {
            engine: Arc::new(engine),
            workspace_id: None,
        })
    }

    /// Index a project
    pub async fn index_project(&mut self, project_root: PathBuf) -> aurora_semantic::Result<()> {
        let ws_config = WorkspaceConfig::new(project_root);
        self.workspace_id = Some(
            self.engine.index_workspace(ws_config, None).await?
        );
        Ok(())
    }

    /// Search code
    pub fn search(&self, query: &str) -> aurora_semantic::Result<Vec<SearchHit>> {
        let workspace_id = self.workspace_id.as_ref()
            .ok_or_else(|| aurora_semantic::Error::config("No workspace indexed"))?;

        let results = self.engine.search_text(workspace_id, query)?;

        Ok(results.into_iter().map(|r| SearchHit {
            file: r.document.relative_path.clone(),
            line: r.chunk.start_line,
            symbol: r.chunk.symbol_name.clone(),
            score: r.score,
            preview: r.chunk.content.chars().take(200).collect(),
        }).collect())
    }
}

pub struct SearchHit {
    pub file: PathBuf,
    pub line: u32,
    pub symbol: Option<String>,
    pub score: f32,
    pub preview: String,
}
```

## Using ONNX Models

### Downloading Models

Aurora works with any ONNX embedding model. Recommended for code:

**Option 1: Download pre-exported ONNX models directly**

```bash
# Create model directory
mkdir -p models/jina-code

# Download Jina Code model files
curl -L -o models/jina-code/model.onnx \
    "https://huggingface.co/jinaai/jina-embeddings-v2-base-code/resolve/main/onnx/model.onnx"
curl -L -o models/jina-code/tokenizer.json \
    "https://huggingface.co/jinaai/jina-embeddings-v2-base-code/resolve/main/tokenizer.json"
```

**Option 2: Export with optimum (for other models)**

```bash
# Install optimum for exporting
pip install optimum[exporters]

# Export Jina Code model (best for code search)
optimum-cli export onnx \
    --model jinaai/jina-embeddings-v2-base-code \
    --task feature-extraction \
    ./models/jina-code
```

### Model Configuration

```rust
use aurora_semantic::{ModelConfig, OnnxEmbedder};

// Simple loading
let embedder = OnnxEmbedder::from_directory("./models/jina-code")?;

// With custom settings
let embedder = ModelConfig::from_directory("./models/jina-code")
    .with_max_length(8192)      // Max tokens (Jina supports 8k)
    .with_dimension(768)         // Embedding dimension
    .load()?;

// Create engine with embedder
let engine = Engine::with_embedder(config, embedder)?;
```

### Checking Model Compatibility

```rust
fn validate_model_dir(path: &Path) -> bool {
    let model_file = path.join("model.onnx");
    let tokenizer_file = path.join("tokenizer.json");

    model_file.exists() && tokenizer_file.exists()
}
```

## Custom Embedders

Implement the `Embedder` trait for custom embedding providers:

```rust
use aurora_semantic::{Embedder, Result, Error};

/// Example: GPU-accelerated embedder
pub struct GpuEmbedder {
    dimension: usize,
    // Your GPU resources here
}

impl Embedder for GpuEmbedder {
    fn embed(&self, text: &str) -> Result<Vec<f32>> {
        // Single text embedding
        self.embed_batch(&[text])
            .map(|v| v.into_iter().next().unwrap())
    }

    fn embed_batch(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
        // Batch embedding (more efficient)
        // Your GPU inference code here

        Ok(texts.iter().map(|_| vec![0.0; self.dimension]).collect())
    }

    fn dimension(&self) -> usize {
        self.dimension
    }

    fn name(&self) -> &'static str {
        "gpu-embedder"
    }

    fn max_sequence_length(&self) -> usize {
        512  // Override if your model supports more
    }
}

// Use it
let embedder = GpuEmbedder { dimension: 768 };
let engine = Engine::with_embedder(config, embedder)?;
```

## Search API

### Basic Search

```rust
use aurora_semantic::{SearchQuery, SearchMode};

// Simple text search
let results = engine.search_text(&workspace_id, "error handling")?;

// With options
let query = SearchQuery::new("authentication")
    .mode(SearchMode::Hybrid)  // Lexical, Semantic, or Hybrid
    .limit(50)
    .min_score(0.3);

let results = engine.search(&workspace_id, query)?;
```

### Filtered Search

```rust
use aurora_semantic::{SearchQuery, SearchFilter, Language, ChunkType};

let query = SearchQuery::new("database")
    .filter(SearchFilter::new()
        // Filter by language
        .languages(vec![Language::Rust, Language::Go])

        // Filter by code construct type
        .chunk_types(vec![ChunkType::Function, ChunkType::Struct])

        // Filter by path patterns
        .path_patterns(vec!["**/src/**".into()])

        // Filter by symbol name
        .symbol_names(vec!["connect".into(), "query".into()])

        // Include/exclude directories
        .in_directories(vec!["src/db".into()])
        .exclude_directories(vec!["tests".into()]));

let results = engine.search(&workspace_id, query)?;
```

### Processing Results

```rust
for result in results {
    println!("File: {}", result.document.relative_path.display());
    println!("Lines: {}-{}", result.chunk.start_line, result.chunk.end_line);
    println!("Type: {:?}", result.chunk.chunk_type);
    println!("Symbol: {:?}", result.chunk.symbol_name);
    println!("Score: {:.2}", result.score);
    println!("Match: {:?}", result.match_type);

    // Highlighted matches
    for highlight in &result.highlights {
        println!("  Matched: '{}' at {}-{}",
            highlight.text, highlight.start, highlight.end);
    }

    // Code preview
    println!("Code:\n{}", result.chunk.content);
}
```

## Progress Reporting

### With Callback

```rust
use aurora_semantic::IndexProgress;

let progress_callback = Box::new(|progress: IndexProgress| {
    let pct = progress.percentage();

    match progress.phase {
        IndexPhase::Scanning => {
            println!("[{:.0}%] Scanning files...", pct);
        }
        IndexPhase::Parsing => {
            if let Some(file) = &progress.current_file {
                println!("[{:.0}%] Parsing: {}", pct, file.display());
            }
        }
        IndexPhase::Embedding => {
            println!("[{:.0}%] Generating embeddings {}/{}",
                pct, progress.processed, progress.total);
        }
        IndexPhase::Indexing => {
            println!("[{:.0}%] Building search index...", pct);
        }
        IndexPhase::Persisting => {
            println!("[{:.0}%] Saving to disk...", pct);
        }
        IndexPhase::Complete => {
            println!("[100%] Indexing complete!");
        }
    }
});

engine.index_workspace(ws_config, Some(progress_callback)).await?;
```

### With Channel (for async UI updates)

```rust
use tokio::sync::mpsc;

let (tx, mut rx) = mpsc::channel(100);

let progress_callback = Box::new(move |progress: IndexProgress| {
    let _ = tx.try_send(progress);
});

// In another task
tokio::spawn(async move {
    while let Some(progress) = rx.recv().await {
        // Update UI
        update_progress_bar(progress.percentage());
    }
});
```

## Workspace Management

### Lifecycle

```rust
// List all indexed workspaces
let workspaces = engine.list_workspaces()?;

for ws_id in &workspaces {
    // Get statistics
    let stats = engine.get_workspace_stats(ws_id)?;

    println!("Workspace: {}", ws_id);
    println!("  Path: {}", stats.root_path.display());
    println!("  Documents: {}", stats.document_count);
    println!("  Chunks: {}", stats.chunk_count);
    println!("  Size: {} KB", stats.total_bytes / 1024);
    println!("  Updated: {}", stats.updated_at);
}

// Load an existing workspace
engine.load_workspace(&workspaces[0])?;

// Delete a workspace
engine.delete_workspace(&workspaces[0])?;
```

### Persistence

Indexes are automatically persisted to the configured `index_dir`:

```
.aurora/
├── <workspace-uuid>/
│   ├── metadata.json       # Workspace metadata
│   ├── documents.bin       # Serialized documents
│   ├── chunks.bin          # Serialized chunks
│   ├── embeddings.bin      # Embedding vectors
│   ├── tantivy/            # Lexical search index
│   └── vectors.usearch     # Vector similarity index
```

## Best Practices

### 1. Model Selection

- **For code search**: Use `jina-embeddings-v2-base-code` (768 dim, 8k context)
- **For general text**: Use `all-MiniLM-L6-v2` (384 dim, 512 context)
- **For speed**: Use smaller models or hash embeddings

### 2. Chunk Size Configuration

```rust
let config = EngineConfig::new(index_dir)
    .with_chunking(ChunkingConfig {
        max_chunk_size: 2000,  // Larger for semantic search
        min_chunk_size: 50,    // Avoid tiny fragments
        extract_comments: true,
    });
```

### 3. Search Weights

```rust
// For keyword-heavy queries (exact matches)
.with_search(SearchConfig {
    lexical_weight: 0.7,
    semantic_weight: 0.3,
    ..Default::default()
})

// For conceptual queries (similar meaning)
.with_search(SearchConfig {
    lexical_weight: 0.2,
    semantic_weight: 0.8,
    ..Default::default()
})
```

### 4. Memory Management

```rust
// For large codebases, configure memory limits
.with_performance(PerformanceConfig {
    num_threads: 4,
    memory_limit: 256 * 1024 * 1024,  // 256MB
    incremental: true,
})
```

## Troubleshooting

### Model Loading Fails

```
Error: model.onnx not found
```

Ensure your model directory contains both `model.onnx` and `tokenizer.json`.

### Out of Memory

Reduce batch size or use a smaller model:

```rust
.with_embedding(EmbeddingConfig {
    batch_size: 8,  // Reduce from default 32
    ..Default::default()
})
```

### Slow Indexing

1. Use hash embeddings for initial testing
2. Enable parallel processing
3. Exclude unnecessary directories

```rust
.with_ignore(IgnoreConfig {
    ignored_directories: vec![
        "node_modules".into(),
        "target".into(),
        ".git".into(),
        "dist".into(),
    ],
    ..Default::default()
})
```

### Search Returns No Results

1. Check workspace is loaded: `engine.load_workspace(&id)?`
2. Lower `min_score` threshold
3. Try `SearchMode::Lexical` first for debugging

## API Reference

See the [rustdoc documentation](https://docs.rs/aurora-semantic) for complete API reference.
