Documentation

Complete guide to setting up and using LLM Link with 10 major LLM providers. Universal proxy service with hot-reload configuration and multiple API format support.

Installation

Option 1: Install from crates.io (Recommended)

cargo install llm-link

Installs the latest stable version from Rust package registry

Option 2: Build from source

git clone https://github.com/lipish/llm-link.git
cd llm-link
cargo build --release

Get the latest features from the main branch

Quick Start

Start with Application Presets

Optimized configurations for popular AI coding tools:

# For Codex CLI (GitHub Copilot CLI)

llm-link --app codex-cli


# For Zed editor

llm-link --app zed


# For Claude Code

llm-link --app claude-code


# For Continue.dev

llm-link --app continue

Start with Specific Protocols

Choose which API protocols to enable:

# Enable specific protocols

llm-link --protocols openai,anthropic,ollama


# Enable all protocols

llm-link --protocols all


# Start on custom port

llm-link --port 8088 --protocols openai,anthropic

Optional API Key Startup

New in v0.3.3: LLM Link can now start without API keys!

Service starts normally and displays warnings for missing keys. Configure API keys later using hot-reload API.

Provider Configuration

Environment Variables

Set API keys as environment variables. For Ollama, no API key is required.

OpenAI

OPENAI_API_KEY

Leading AI models including GPT-4, GPT-3.5, and more

Streaming Function Calling Vision

Anthropic

ANTHROPIC_API_KEY

Advanced Claude models with strong reasoning capabilities

Streaming Long Context Vision

Zhipu AI

ZHIPU_API_KEY

Chinese AI models with multilingual support

Streaming Multilingual Code Generation

Aliyun

ALIYUN_API_KEY

Alibaba Cloud's powerful Qwen models

Streaming Long Context Multilingual

Volcengine

VOLCENGINE_API_KEY

ByteDance's advanced Doubao models

Streaming Cost Effective Fast Response

Tencent

TENCENT_API_KEY

Tencent's Hunyuan models for various applications

Streaming Chinese Optimized Enterprise Ready

Longcat

LONGCAT_API_KEY

High-performance models for general dialogue

Streaming Fast Response Cost Effective

Moonshot

MOONSHOT_API_KEY

Kimi models with large context windows

Streaming 200K Context Document Processing

Minimax

MINIMAX_API_KEY

Powerful AI models with OpenAI-compatible API

Streaming Multilingual Fast Response

Ollama

None Required

Local and open-source models

Local Deployment Privacy Custom Models

Configuration File (Optional)

Create a keys.yaml file in the project directory:

providers:

 openai:

  api_key: "your_openai_key"

 anthropic:

  api_key: "your_anthropic_key"

 zhipu:

  api_key: "your_zhipu_key"

 # ... other providers

Hot Reload Configuration

Dynamic Configuration Update

Update provider configurations without restarting the service using the REST API:

curl -X POST http://localhost:8088/api/config/update \
  -H "Content-Type: application/json" \
  -d '{"provider": "openai", "api_key": "new_key"}'

Available API Endpoints

Update Configuration

POST /api/config/update

Update provider API keys and settings

List Models

GET /api/models

Get available models for all providers

Provider Status

GET /api/providers

Check status of all configured providers

Health Check

GET /api/health

Service health and version information

API Protocols

OpenAI API

Compatible with OpenAI's API format

Endpoint: /v1

Supports chat completions, streaming, function calling

Anthropic API

Native Anthropic Claude API

Endpoint: /v1/messages

Supports messages API, streaming, long context

Ollama API

Compatible with Ollama's API format

Endpoint: /api/generate

Supports local models, custom models, streaming

As a Rust Library

Add to Cargo.toml

[dependencies]
llm-link = "0.3.5"

Usage Examples

List All Providers

use llm_link::provider::ProviderRegistry;

// List all available providers
let providers = ProviderRegistry::list_providers();
println!("Available providers: {:?}", providers)

Get Models for Provider

use llm_link::models::ModelsConfig;

// Load models configuration
let models = ModelsConfig::load_with_fallback()
    .get_models_for_provider("openai");

for model in models {
    println!("Model: {}", model.name);
}

Create LLM Client

use llm_link::provider::{Provider, ProviderConfig};
use llm_link::provider::openai::OpenAIProvider;

let config = ProviderConfig {
    api_key: "your_key".to_string(),
    base_url: None,
};

let client = OpenAIProvider::create_client(&config)?;
// Use client for chat completions

Troubleshooting

Service fails to start

Check if required ports are available. Use --port to specify a different port.

API key not working

Verify API key format and permissions. Use hot-reload API to update keys without restart.

Ollama connection failed

Ensure Ollama is running on localhost:11434. Use ollama pull to download models.

View on GitHub API Reference View All Providers