# squeez

[![CI](https://github.com/claudioemmanuel/squeez/actions/workflows/ci.yml/badge.svg)](https://github.com/claudioemmanuel/squeez/actions/workflows/ci.yml)
[![npm](https://img.shields.io/npm/v/squeez.svg)](https://www.npmjs.com/package/squeez)
[![Crates.io](https://img.shields.io/crates/v/squeez.svg)](https://crates.io/crates/squeez)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Changelog](https://img.shields.io/badge/changelog-📋-blue.svg)](CHANGELOG.md)

End-to-end token optimizer for Claude Code, GitHub Copilot CLI, OpenCode, Gemini CLI, and OpenAI Codex CLI. Compresses bash output up to **95%**, collapses redundant calls, and injects a terse prompt persona — automatically, with zero new runtime dependencies.

---

## Install

Three methods — all produce the same result (binary at `~/.claude/squeez/bin/squeez`, hooks registered).

### curl (recommended)

```bash
curl -fsSL https://raw.githubusercontent.com/claudioemmanuel/squeez/main/install.sh | sh
```

> **Windows:** requires [Git Bash](https://git-scm.com/downloads). Run the command above inside Git Bash — PowerShell/CMD are not supported.

### npm / npx

```bash
# Install globally
npm install -g squeez

# Or run once without installing
npx squeez
```

Downloads the correct pre-built binary for your platform (macOS universal, Linux x86_64/aarch64, Windows x86_64). Requires Node ≥ 16.

### cargo (build from source)

```bash
cargo install squeez
```

Builds from [crates.io](https://crates.io/crates/squeez). Requires Rust stable. On Windows you also need [MSVC C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/).

---

### Supported hosts

`squeez setup` auto-detects every CLI present on disk and registers the hooks. `squeez uninstall` removes them. Session data and `config.ini` are preserved so reinstall is lossless.

| Host | Memory file | Bash wrap | Session memory | Budget inject (Read/Grep) | Notes |
|---|---|---|---|---|---|
| **Claude Code** | `~/.claude/CLAUDE.md` | ✅ native | ✅ native | ✅ native | Restart Claude Code to pick up hooks |
| **Copilot CLI** | `~/.copilot/copilot-instructions.md` | ✅ native | ✅ native | ✅ native | Restart Copilot CLI after setup |
| **OpenCode** | `~/.config/opencode/AGENTS.md` | ✅ native | ✅ native | ✅ native | Plugin at `~/.config/opencode/plugins/squeez.js`; MCP tool calls skip hooks (upstream sst/opencode#2319) |
| **Gemini CLI** | `~/.gemini/GEMINI.md` | ✅ native | ✅ native | 🟡 soft via `GEMINI.md` | `BeforeTool` rewrite schema pending upstream docs ([google-gemini/gemini-cli#25629](https://github.com/google-gemini/gemini-cli/issues/25629)) |
| **Codex CLI** | `~/.codex/AGENTS.md` | ✅ native | ✅ native | 🟡 soft via `AGENTS.md` | Codex `PreToolUse` is Bash-only until upstream expands ([openai/codex#18491](https://github.com/openai/codex/issues/18491)) |

### Manage

```bash
squeez setup                  # register into every detected host
squeez setup --host=<slug>    # register into one host
squeez uninstall              # remove squeez entries from every detected host
squeez uninstall --host=<slug>
```

Slugs: `claude-code` / `copilot` / `opencode` / `gemini` / `codex`.

After install, restart the CLI you use to pick up the new hooks.

### Uninstall

```bash
squeez uninstall              # preserves session data + config.ini
bash ~/.claude/squeez/uninstall.sh   # (legacy) full wipe, if the script exists
```

### Self-update

```bash
squeez update             # download latest binary + verify SHA256
squeez update --check     # check for update without installing
squeez update --insecure  # skip checksum (not recommended)
```

---

## What it does

| Feature | Description |
|---------|-------------|
| **Bash compression** | Intercepts every command via `PreToolUse` hook, applies smart filter → dedup → grouping → truncation. Up to 95% reduction. |
| **Context engine** | Cross-call redundancy with two paths: exact-hash match (FNV-1a, fast) **and** fuzzy trigram-shingle Jaccard ≥0.85 (whitespace, timestamps, single-line edits no longer defeat dedup). |
| **Summarize fallback** | Outputs exceeding 500 lines are replaced with a ≤40-line dense summary (top errors, files, test result, tail). **Benign outputs get 2× the threshold** so successful builds stay verbatim. |
| **Adaptive intensity** | Truly adaptive: **Full** (×0.6 limits) below 80% of token budget, **Ultra** (×0.3) above. Used to be always-Ultra; now actually responds to session pressure. |
| **MCP server** | `squeez mcp` runs a JSON-RPC 2.0 server over stdio exposing 13 read-only tools so any MCP-compatible LLM can query session memory directly. Hand-rolled, no `mcp.server` dependency. |
| **Auto-teach payload** | `squeez protocol` (or the `squeez_protocol` MCP tool) prints a 2.4 KB self-describing payload — the LLM learns squeez's markers and protocol on first call. |
| **Caveman persona** | Injects an ultra-terse prompt at session start so the model responds with fewer tokens. |
| **Memory-file compression** | `squeez compress-md` compresses CLAUDE.md / AGENTS.md / copilot-instructions.md in-place — pure Rust, zero LLM. i18n-aware: set `lang = pt` (or `--lang pt`) for pt-BR article/filler/phrase dropping and Unicode-correct matching. |
| **Session memory** | On `SessionStart`, injects a structured summary of the previous session: files investigated, learned facts (errors + git events), completed work (builds, test passes), and next steps (unresolved errors, failing tests). Summaries carry temporal validity (`valid_from`/`valid_to`). |
| **Token tracking** | Every `PostToolUse` result (Bash, Read, Grep, Glob) feeds a `SessionContext` so squeez knows what the agent has already seen. |
| **Token economy** | Sub-agent cost tracking (~200K tokens/spawn), burn rate prediction (`[budget: ~N calls left]`), session efficiency scoring, tool result size budgets. |
| **Auto-calibration** | `squeez calibrate` runs benchmarks on install and generates an optimized `config.ini` (aggressive / balanced / conservative profiles). |

---

## Benchmarks

<!-- BENCHMARK:START -->
Measured on macOS (Apple Silicon). Token count = `chars / 4` (matches Claude's ~4 chars/token). Run `squeez benchmark` to reproduce.

### Per-scenario results — 23 scenarios × 5 iterations

| Scenario | Before | After | Reduction | Latency |
|----------|--------|-------|-----------|---------|
| `summarize_huge` | 82,257 tk | 420 tk | **-99%** | 55.7 ms |
| `repetitive_output` | 4,692 tk | 37 tk | **-99%** | 233 µs |
| `high_context_adaptive` | 4,418 tk | 52 tk | **-99%** | 835 µs |
| `ps_aux` | 40,373 tk | 2,352 tk | **-94%** | 3.0 ms |
| `git_log_200` | 2,692 tk | 289 tk | **-89%** | 220 µs |
| `tsc_errors` | 731 tk | 101 tk | **-86%** | 29 µs |
| `cargo_build_noisy` | 2,106 tk | 452 tk | **-79%** | 262 µs |
| `docker_logs` | 665 tk | 186 tk | **-72%** | 47 µs |
| `find_deep` | 424 tk | 134 tk | **-68%** | 79 µs |
| `git_status` | 50 tk | 16 tk | **-68%** | 12 µs |
| `state_first_simulation` | 182 tk | 69 tk | **-62%** | 12 µs |
| `verbose_app_log` | 4,957 tk | 1,991 tk | **-60%** | 315 µs |
| `npm_install` | 524 tk | 232 tk | **-56%** | 49 µs |
| `claude_md_overhead` | 717 tk | 318 tk | **-56%** | 361 µs |
| `crosscall_redundancy_3x` | 486 tk | 241 tk | **-50%** | 51.5 ms |
| `ls_la` | 1,782 tk | 886 tk | **-50%** | 218 µs |
| `env_dump` | 441 tk | 287 tk | **-35%** | 25 µs |
| `git_copilot` | 640 tk | 421 tk | **-34%** | 104 µs |
| `agent_heavy` | 2,306 tk | 1,564 tk | **-32%** | 408 µs |
| `md_prose` | 187 tk | 138 tk | **-26%** | 678 µs |
| `md_claude_md` | 316 tk | 247 tk | **-22%** | 1.3 ms |
| `git_diff` | 502 tk | 497 tk | **-1%** | 43 µs |
| `kubectl_pods` | 1,513 tk | 1,513 tk | **-0%** | 31 µs |

### Aggregate

| Metric | Value |
|--------|-------|
| **Total token reduction** | **91.9%** — 152,961 tk → 12,443 tk |
| Bash output | **-84.9%** |
| Markdown / context files | **-23.5%** |
| Wrap / cross-call engine | **-99.2%** |
| Quality (signal terms preserved) | **23 / 23 pass** |
| Latency p50 (filter mode) | **5.0 ms** |
| Latency p95 (incl. wrap/summarize) | **51 ms** |

### Estimated cost savings — Claude Sonnet 4.6 · $3.00 / MTok input

| Usage | Baseline / month | Saved / month |
|-------|-----------------|---------------|
| 100 calls / day | $18.00 | **$16.54 (92%)** |
| 1,000 calls / day | $180.00 | **$165.37 (92%)** |
| 10,000 calls / day | $1800.00 | **$1653.66 (92%)** |
<!-- BENCHMARK:END -->

---

## Commands

```bash
squeez wrap <cmd>                        # compress a command's output end-to-end
squeez filter <hint>                     # compress stdin (piped usage)
squeez compress-md [--ultra] [--dry-run] [--all] <file>...   # compress markdown files
squeez benchmark [--json] [--output <file>] [--scenario <name>] [--iterations <n>]
squeez mcp                               # JSON-RPC 2.0 MCP server over stdin/stdout
squeez protocol                          # print the auto-teach payload (markers + protocol)
squeez update [--check] [--insecure]     # self-update
squeez init [--copilot]                  # session-start hook (called by hook, not manually)
squeez calibrate                         # auto-tune config from benchmarks
squeez budget-params <tool>              # output JSON budget patch for tool
squeez --version
```

### Escape hatch — bypass compression for one command

```bash
--no-squeez git log --all --graph
```

Prefix any command with `--no-squeez` to run it raw without squeez touching it.

### `squeez wrap`

Runs a command, compresses its output, and prints a savings header:

```
# squeez [git log] 2692→289 tokens (-89%) 0.2ms [adaptive: Ultra]
```

### `squeez filter`

Reads from stdin. Use for manual pipelines:

```bash
git log --oneline | squeez filter git
docker logs mycontainer 2>&1 | squeez filter docker
```

### `squeez compress-md`

Pure-Rust, zero-LLM compressor for markdown files. Preserves code blocks, inline code, URLs, headings, file paths, and tables. Compresses prose only. Always writes a backup at `<stem>.original.md`.

```bash
squeez compress-md CLAUDE.md             # Full mode (English default)
squeez compress-md --ultra CLAUDE.md    # + abbreviations (with→w/, fn, cfg, etc.)
squeez compress-md --lang pt CLAUDE.md  # pt-BR locale (articles, fillers, phrases)
squeez compress-md --dry-run CLAUDE.md  # preview, no write
squeez compress-md --all                # compress all known locations automatically
```

When `auto_compress_md = true` (default), `squeez init` runs `--all` silently on every session start.

### `squeez benchmark`

Reproducible measurement of token reduction, cost, latency, and quality across 19 scenarios:

```bash
squeez benchmark                          # human-readable report
squeez benchmark --json                   # JSON to stdout
squeez benchmark --output report.json     # save JSON report
squeez benchmark --scenario git           # run only git scenarios
squeez benchmark --iterations 5           # more iterations per scenario
squeez benchmark --list                   # list all scenarios
```

Quality is scored by checking that **signal terms** (words from error/warning/failed lines in the baseline) survive compression. 19/19 pass at ≥ 50% threshold.

### `squeez mcp`

Runs a Model Context Protocol JSON-RPC 2.0 server over stdin/stdout. Hand-rolled, no `mcp.server` / `fastmcp` dependency — keeps the `libc`-only constraint intact. Wire it into Claude Code:

```bash
claude mcp add squeez -- /path/to/squeez mcp
```

Thirteen read-only tools become available to the LLM:

| Tool | Returns |
|------|---------|
| `squeez_recent_calls` | Last N bash invocations with hash + length + cmd snippet — check before re-running |
| `squeez_seen_files` | Files this session has touched, with access type (Read/Write/Created/Deleted), sorted by recency |
| `squeez_seen_errors` | Distinct error fingerprints observed this session (FNV-1a hashes of normalized errors) |
| `squeez_seen_error_details` | Error fingerprints with the first 128 chars of message text — find *what* the error was |
| `squeez_session_summary` | Token accounting + call counts (tokens_bash / tokens_read / tokens_other / seen_files / seen_errors / seen_git_refs) |
| `squeez_session_stats` | Dedup hit counts (exact + fuzzy), summarize triggers, Ultra-mode calls, tokens saved per category |
| `squeez_agent_costs` | Sub-agent usage: spawn count, cumulative estimated tokens, per-call breakdown |
| `squeez_session_efficiency` | Session efficiency scores: compression ratio, tool choice, context reuse, budget conservation (basis points) |
| `squeez_prior_summaries` | Last N finalized prior-session summaries with structured fields: investigated / learned / completed / next_steps |
| `squeez_search_history` | Full-text search across all session summaries — find when you last saw an error or touched a file |
| `squeez_file_history` | Sessions where a given file path was touched, with token-savings and commit status |
| `squeez_session_detail` | Full structured view of a past session by date: calls, files, errors, git events, test summary |
| `squeez_protocol` | Auto-teach payload — read once per session to learn squeez's markers + memory protocol |

All read-only. Backed by `SessionContext::load()`, `memory::read_last_n()`, and `memory::search_history()`. No side effects.

### `squeez protocol`

Prints the auto-teach payload — a 2.4 KB self-describing block covering:

- The 5-rule **memory protocol** (what to do with `[squeez: ...]` markers, when to call the MCP tools)
- The **output marker spec** (`# squeez [...]`, `[squeez: identical to ...]`, `[squeez: ~95% similar to ...]`, `squeez:summary`, `# squeez hint:`)

Same content the MCP `squeez_protocol` tool returns. Pipe it into a `system` prompt or paste it into a one-shot session that doesn't have the MCP server connected.

---

## Configuration

Optional config file — all fields have defaults, none are required.

| Platform | Config path |
|----------|------------|
| Claude Code / default | `~/.claude/squeez/config.ini` |
| Copilot CLI | `~/.copilot/squeez/config.ini` |

```ini
# ── Compression ────────────────────────────────────────────────
max_lines              = 200     # generic truncation limit
dedup_min              = 3       # collapse lines appearing ≥N times
git_log_max_commits    = 20
git_diff_max_lines     = 150
docker_logs_max_lines  = 100
find_max_results       = 50
bypass                 = docker exec, psql, mysql, ssh   # never compress these

# ── Context engine ─────────────────────────────────────────────
adaptive_intensity         = true    # truly adaptive: Full <80% budget, Ultra ≥80%
context_cache_enabled      = true    # track seen files/errors across calls
redundancy_cache_enabled   = true    # collapse identical OR fuzzy-similar recent outputs
summarize_threshold_lines  = 500     # outputs above this trigger summarize fallback (×2 if benign)
compact_threshold_tokens   = 120000  # session token budget — drives adaptive intensity

# ── Session memory ─────────────────────────────────────────────
memory_retention_days = 30

# ── Output / persona ───────────────────────────────────────────
persona          = ultra    # off | lite | full | ultra
auto_compress_md = true     # run compress-md on every session start
lang             = en       # compress-md locale: en | pt (pt-BR) — more languages extensible

# ── Advanced tuning (rarely needed) ───────────────────────────
max_call_log              = 32    # rolling call log depth (also caps redundancy window)
recent_window             = 16    # how many recent calls are eligible for redundancy lookup
similarity_threshold      = 0.85  # Jaccard threshold for fuzzy dedup (0.0–1.0)
ultra_trigger_pct         = 0.80  # fraction of context budget at which Full → Ultra
mcp_prior_summaries_default = 5   # default n for squeez_prior_summaries
mcp_recent_calls_default    = 10  # default n for squeez_recent_calls

# ── Token economy ─────────────────────────────────────────────
agent_warn_threshold_pct  = 0.50  # warn when agent cost > 50% of budget
burn_rate_warn_calls      = 20    # warn when < 20 calls remaining
agent_spawn_cost          = 200000 # estimated tokens per Agent/Task spawn
read_max_lines            = 0     # max lines injected into Read tool_input (0 = off)
grep_max_results          = 0     # max results injected into Grep tool_input (0 = off)
```

### Adaptive intensity — Full / Ultra split

When `adaptive_intensity = true` (default), squeez **actually adapts** to session pressure rather than always running Ultra:

| Used / budget | Tier | Scaling |
|---|---|---|
| `< 80%` | **Full** | ×0.6 limits, dedup_min ×0.66 (floor 2) |
| `≥ 80%` | **Ultra** | ×0.3 limits, dedup_min ×0.5 (floor 2) |
| `adaptive_intensity = false` | **Lite** | passthrough — no scaling |

Floors are enforced so we never reduce to zero: `max_lines ≥ 20`, `git_diff_max_lines ≥ 20`, `dedup_min ≥ 2`, `summarize_threshold_lines ≥ 50`.

The active level is shown in every bash header: `[adaptive: Full]` or `[adaptive: Ultra]`.

Pre-0.3 squeez was effectively always-Ultra. The new behavior preserves more verbatim text in the common case (empty / mid-session) and only graduates to aggressive compression when the context budget is genuinely under pressure.

### Caveman persona

Three intensity levels (`lite`, `full`, `ultra`) and `off`. Default is `ultra`. The persona prompt is injected into:
- The Claude Code session banner (printed at `SessionStart`)
- The `<!-- squeez:start -->…<!-- squeez:end -->` block in `~/.copilot/copilot-instructions.md` for Copilot CLI

---

## How it works

### Compression pipeline

Each bash command passes through four strategies in order:

1. **smart_filter** — strips ANSI codes, progress bars, spinner chars, timestamps, and tool-specific noise (npm download lines, stack frame noise, etc.)
2. **dedup** — lines appearing ≥ `dedup_min` times are collapsed to one entry annotated `[×N]`
3. **grouping** — files in the same directory (≥5 siblings) are collapsed to `dir/  N modified  [squeez grouped]`
4. **truncation** — `Head` (keep first N) or `Tail` (keep last N) depending on handler; truncated portion noted

### Supported handlers

| Category | Commands |
|----------|----------|
| Git | `git` |
| Docker / containers | `docker`, `docker-compose`, `podman` |
| Package managers | `npm`, `pnpm`, `bun`, `yarn` |
| Build systems | `make`, `cmake`, `gradle`, `mvn`, `xcodebuild`, `cargo` (build), `next build/dev/start` |
| Test runners | `cargo test`, `jest`, `vitest`, `pytest`, `nextest`, `playwright`, `bun test` |
| TypeScript / linters | `tsc`, `eslint`, `biome` |
| Cloud CLIs | `kubectl`, `gh`, `aws`, `gcloud`, `az`, `wrangler` |
| Databases | `psql`, `prisma`, `mysql`, `drizzle-kit` |
| Filesystem | `find`, `ls`, `du`, `ps`, `env`, `lsof`, `netstat` |
| JSON / YAML / IaC | `jq`, `yq`, `terraform`, `tofu`, `helm`, `pulumi` |
| Text processing | `grep`, `rg`, `awk`, `sed` |
| Network | `curl`, `wget` |
| Runtimes | `node`, `python`, `ruby` |
| Generic fallback | everything else |

### Hooks (Claude Code & Copilot CLI)

Three hooks work together automatically after install:

- **`PreToolUse`** — rewrites every Bash call: `git status` → `squeez wrap git status`
- **`SessionStart`** — runs `squeez init`: finalizes previous session into a memory summary, injects the persona prompt
- **`PostToolUse`** — runs `squeez track-result`: scans every tool result (Bash, Read, Grep, Glob) for file paths and errors, feeding `SessionContext`

### Cross-call redundancy

Two-path dedup across the last 16 calls:

**Exact match** — FNV-1a hash of the compressed output. When a subsequent call produces the same bytes, it collapses to:

```
[squeez: identical to 515ba5b2 at bash#35 — re-run with --no-squeez]
```

**Fuzzy match** — bottom-k MinHash over whitespace-token trigrams (k=96, Jaccard ≥ 0.85, length-ratio guard ≥ 0.80). Survives timestamp changes, added/removed blank lines, and single-line edits. Collapses to:

```
[squeez: ~92% similar to 515ba5b2 at bash#35 — re-run with --no-squeez]
```

Minimum 6 lines to attempt fuzzy match (below that, exact-only).

### Summarize fallback

When raw output exceeds `summarize_threshold_lines` (default 500), the full pipeline is bypassed and replaced with a ≤40-line dense summary:

```
squeez:summary cmd=docker logs app
total_lines=5003
top_errors:
  - error: connection refused on tcp://10.0.0.1:5432
top_files:
  - /var/log/app/error.log
test_summary=FAILED: 3 of 248
tail_preserved=20
[last 20 lines verbatim...]
```

**Benign-aware threshold:** before summarizing, squeez scans for error markers (`error:`, `panic`, `traceback`, `FAILED`, `EXCEPTION`, `Fatal`). If none are found, the threshold is doubled (1,000 lines default) so successful builds, clean test runs, and uneventful logs stay verbatim unless they are genuinely huge.

---

## Platform notes

### OpenCode

Plugin installed at `~/.config/opencode/plugins/squeez.js`. OpenCode auto-loads plugins on startup. All Bash commands are automatically compressed via `squeez wrap`.

### GitHub Copilot CLI

Hooks registered in `~/.copilot/settings.json`. Session memory written to `~/.copilot/copilot-instructions.md` (Copilot CLI reads this automatically). State stored separately at `~/.copilot/squeez/`.

Refresh memory manually:

```bash
SQUEEZ_DIR=~/.copilot/squeez ~/.claude/squeez/bin/squeez init --copilot
```

---

## Local development

Requires Rust stable. Windows requires Git Bash.

```bash
git clone https://github.com/claudioemmanuel/squeez.git
cd squeez

cargo test                  # run all tests (356 tests, 37 suites)
cargo build --release       # build release binary

bash bench/run.sh           # filter-mode benchmark (14 fixtures)
bash bench/run_context.sh   # context-engine benchmark (3 wrap scenarios)
./target/release/squeez benchmark   # full 19-scenario benchmark suite

bash build.sh               # build + install to ~/.claude/squeez/bin/
```

---

## Contributing

```bash
git checkout -b feature/your-change
cargo test
cargo build --release
bash bench/run.sh
git push -u origin feature/your-change
gh pr create --base main --title "Short title" --body "Description"
```

CI runs `cargo test`, `bench/run.sh`, `bench/run_context.sh`, and `squeez benchmark` on every push and pull request.

See [CONTRIBUTING.md](CONTRIBUTING.md) for coding standards.

---

## License

MIT — see [LICENSE](LICENSE).
