# memory-mcp

A semantic memory system for AI coding agents, exposed as an MCP server. Memories are stored as files in a git repository, synced across devices via a private GitHub remote, and indexed for semantic retrieval.

## Problem

Current memory systems (e.g. Serena memories, Claude Code auto-memory) have limitations:

- **No sync** — memories are local files tied to a single machine
- **No semantic search** — retrieval requires knowing exact memory names; you must list all memories then pick by name
- **Coupled to other tools** — memory lifecycle is tied to the coding tool's MCP server
- **No scalable organization** — flat namespace, manual categorization, no way to surface relevant context automatically

## Core idea

Separate memory from coding tools. Build a dedicated MCP server that:

1. Stores memories as files in a **git repository** (the "memory repo")
2. Syncs across devices via **push/pull to a private GitHub repo**
3. Maintains a **local embedding index** for semantic search, rebuilt on pull
4. Exposes high-level operations — `remember`, `recall`, `forget` — rather than low-level file ops

## Key design shift

Instead of:
```
list_memories → scan names → read_memory("exact-name")
```

The interface becomes:
```
recall("what do I know about Rust error handling patterns?")
```

The agent describes what it needs, and the system returns relevant memories ranked by semantic similarity. No need to enumerate or guess filenames.

## Architecture

```
┌─────────────┐     ┌──────────────────┐     ┌─────────────┐
│  AI Agent    │────▶│  memory-mcp      │────▶│  Memory Repo│
│  (Claude)    │◀────│  (MCP server)    │◀────│  (git)      │
└─────────────┘     └──────────────────┘     └──────┬──────┘
                            │                        │
                            ▼                        ▼
                    ┌──────────────┐         ┌──────────────┐
                    │  Embedding   │         │  GitHub      │
                    │  Index       │         │  (private)   │
                    │  (local)     │         │  remote      │
                    └──────────────┘         └──────────────┘
```

### Memory repo

- A regular git repository, separate from any project
- Each memory is a markdown file (or other plain text)
- Directory structure provides coarse organization (e.g. `global/`, `projects/cc-toolgate/`)
- Commits track memory evolution over time
- Push/pull to a private GitHub repo for cross-device sync

### Embedding index

- Built locally from the memory files
- Rebuilt on pull (or incrementally updated)
- Used for semantic search — find memories by meaning, not by name
- Could use a local embedding model or an API-based one
- Stored in the repo (or gitignored and rebuilt) — TBD

### MCP server

Exposes tools to the AI agent:

- **`remember(content, [tags], [scope])`** — store a new memory or update an existing one. The server decides the filename/path, writes the file, commits, and updates the index.
- **`recall(query, [scope], [limit])`** — semantic search across memories. Returns ranked results with content and metadata.
- **`forget(query_or_id)`** — remove a memory. Commits the deletion.
- **`sync()`** — pull from remote, rebuild index, push local changes. Could also run automatically on server start and periodically.

## Open questions

- **Embedding model**: local (e.g. sentence-transformers) vs API (e.g. Anthropic/OpenAI embeddings)? Local is simpler for offline use and privacy. API may give better quality.
- **Conflict resolution**: what happens when two devices edit the same memory? Git merge handles text conflicts, but semantic conflicts (contradictory facts) need a strategy.
- **Memory format**: plain markdown? Frontmatter with metadata (tags, timestamps, source)? Structured YAML?
- **Deduplication**: how to detect when a new `remember` call is updating an existing memory vs creating a new one? Semantic similarity threshold?
- **Scope model**: global memories vs project-scoped vs session-scoped? How does scoping map to directory structure?
- **Index storage**: commit the index to the repo (portable but larger), or gitignore it and rebuild (clean but slower on first pull)?
- **Language/runtime**: Python (rich embedding ecosystem, uvx-friendly) vs Rust (performance, aligns with existing projects)?
- **Auth for sync**: SSH keys? GitHub CLI token? How to bootstrap on a new device?

## Prior art / related

- Serena memories (file-based, no sync, no semantic search)
- Claude Code auto-memory (flat markdown, single machine)
- mem0 / memgpt (agent memory frameworks, but SaaS-oriented)
- Obsidian + git sync (similar git-backed knowledge base pattern)
