# duckduckgo-search-cli

- Rust CLI for DuckDuckGo search — returns results as JSON, Markdown, or plain text
- Install with: `cargo install duckduckgo-search-cli`
- Current version: v0.7.7
- Repository: https://github.com/daniloaguiarbr/duckduckgo-search-cli


## What's new in v0.7.7 (2026-06-14)

- **GAP-WS-49 fixed (CRITICAL, runtime) — real queries returned ZERO results due to detectable TLS fingerprint.** v0.7.6 closed `cargo install` but the published binary passed all smoke tests (`--probe` and `--probe-deep` returned status 200/ok) while real queries silently returned `resultados: 0` with `cascade_level: 0` and `usou_endpoint_fallback: false`. Local reproduction: 6/6 queries returned 0 results with 1.0–1.6s latency.
- **Root cause**: `wreq 6.0.0-rc.29` has no native `emulation` feature — the Chrome/Safari TLS fingerprint emulation lived only in `wreq-util 3.0.0-rc.12` via `default = ["emulation"]`. v0.7.6 removed `wreq-util` (along with the `brotli` feature) to close GAP-WS-48, and without emulation, `wreq 6.0.0-rc.29` with BoringSSL plain produces a TLS handshake whose JA3/JA4 fingerprint is detectable by Cloudflare Bot Management. DDG serves `anomaly-modal` HTML (45 occurrences in the body) to any client that does not present a real browser fingerprint.
- **Cross-confirmation**: direct `curl` with real browser headers (`User-Agent: Chrome/120`, `Accept-Encoding: gzip, deflate, br`, `Cookie: kl=br-pt`, `Sec-Fetch-*`) **ALSO** received `anomaly-modal` at test time (2026-06-14 09:25 UTC), confirming the tightening is upstream and persistent. The minimal 1-request probe (`--probe-deep`) does not trigger tightening because DDG fingerprints by volume/behavior, not by single request.
- **Fix applied**:
  1. Re-added `wreq-util = { version = "3.0.0-rc", default-features = false, features = ["emulation"] }` to `Cargo.toml` (only `emulation`, no `default`, to avoid pulling `brotli` accidentally).
  2. Re-added `"brotli"` feature to `wreq` (required because `wreq-util` `emulation` does `dep:brotli`).
  3. Added 2 direct dep pins to `Cargo.toml` to force compatible versions in `cargo install`:
     - `brotli-decompressor = "=5.0.1"` — versions 5.0.0/5.0.1 have `alloc-no-stdlib = "2.0"` (hard); 5.0.2 published 2026-06-14 widened to `>=2.0.4, <4` and is what pulls 3.0.0 into the graph.
     - `alloc-no-stdlib = "=2.0.4"` — hard pin required because `brotli 8.0.3` mandates `alloc-no-stdlib = "2.0"`.
  4. `cargo update -p alloc-no-stdlib@3.0.0 --precise 2.0.4` removes the 3.0.0 version from the graph (pinning it together is not enough because `cargo install` without `--locked` can resurrect it).
  5. Expanded comment in `Cargo.toml` documenting GAP-WS-49 and the pin strategy.
- **Post-fix validation**:
  - `cargo tree --offline` → graph contains exactly `alloc-no-stdlib v2.0.4` and `brotli-decompressor v5.0.1`, zero 3.0.0/0.2.3 occurrences.
  - `cargo build --release --offline` → **success in 24.04s** (vs 37.14s v0.7.6 — faster because `brotli-decompressor 5.0.1` is smaller than 5.0.2).
  - `cargo install --path .` (without `--locked`, reproducing user path) → **success**, binary installed and functional.
  - Real query `"rust async runtime"` with v0.7.7 binary → **`quantidade_resultados: 5`**, latency 1087ms, real results: `The Async Ecosystem`, `Fundamentals of Asynchronous Programming`, `Tokio - An asynchronous Rust runtime`, etc.
- **Impact**:
  - Final binary: +160KB (brotli 8.0.3 + brotli-decompressor 5.0.1 + wreq-util 3.0.0-rc.12) — accepted trade to restore Chrome/Safari TLS fingerprint and beat DDG anti-bot.
  - `cargo install` build time: ~24s (vs ~37s v0.7.6).
  - Supply chain surface: +3 crates.
  - **Functionality restored**: real queries return 5+ results with a TLS fingerprint identical to real Chrome/Safari.


## What's new in v0.7.6 (2026-06-14)

- **GAP-WS-48 fixed (CRITICAL, install) — `cargo install` broke on 2026-06-14 due to `alloc-no-stdlib 2.0.4 vs 3.0.0` conflict.** On the same day, crates.io released `alloc-no-stdlib 3.0.0`, `alloc-stdlib 0.2.3` (which depends on `alloc-no-stdlib = ">=2.0.4, <4.0.0"`), and `brotli-decompressor 5.0.2`. `cargo install <crate>@<version>` (without `--locked`) regenerates the lock on the destination system and lands on the new versions; `brotli 8.0.3` (not updated, still hard-requires `alloc-no-stdlib = "2.0"`) implements `impl BrotliAlloc for StandardAlloc` expecting the 2.0.4 trait, but `StandardAlloc` from `alloc-stdlib 0.2.3` is compiled against 3.0.0 — collision produces 36 `E0277` errors at `enc/reader.rs`, `enc/writer.rs`, `enc/combined_alloc.rs`.
- **Root cause in 2 layers**: (1) `wreq-util 3.0.0-rc.12` (declared as a direct dep, NEVER imported in `src/`) has `default = ["emulation"]` which enables `dep:brotli` — this was the real brotli carrier in the production dep graph, not the `wreq` brotli feature. (2) The `wreq` `brotli` feature was kept despite DuckDuckGo never sending `Content-Encoding: br` (verified 2026-06-14 against homepage, `/html/`, `/lite/` via `curl -I`).
- **Fix applied**: removed `wreq-util = "3.0.0-rc"` direct dep (was dead code) and removed the `brotli` feature from `wreq`. `gzip`+`deflate`+`zstd` still enabled; DuckDuckGo never sends brotli, so zero functional loss.
- **Validation**: `cargo tree --offline | rg 'brotli|alloc-no-stdlib|alloc-stdlib|wreq-util'` → 0 matches. `cargo install --path . --offline` (without `--locked`, the user-facing path) → success in 35.7s. `cargo build --release` → success in 37.14s (5.92s faster than v0.7.5 by not compiling brotli).
- **Impact**: 6 fewer crates in the dep graph, ~5-10s faster `cargo install`, smaller binary surface. No CLI changes, no JSON schema changes.


## What's new in v0.7.4

- **GAP-WS-28 fixed (build experience, Windows).** `cargo install` on native Windows MSVC without the NASM assembler previously failed minutes into the BoringSSL build with the cryptic `CMake Error: No CMAKE_ASM_NASM_COMPILER could be found`. A new `build.rs` preflight now fails in SECONDS with the exact fix (`winget install -e --id NASM.NASM` + PATH adjustment). Escape hatch: `DDG_SKIP_NASM_CHECK=1`. Root cause: the `btls-sys` v0.5.6 branch that sets `OPENSSL_NO_ASM=YES` for Windows is unreachable in native builds (early return when host == target).
- **New helper `scripts/install-windows.ps1`** — detects NASM, installs it via winget (choco fallback), fixes the session PATH, and runs `cargo install duckduckgo-search-cli --locked`.
- **CI hardening** — Windows jobs in `ci.yml` and `release.yml` verify/install NASM explicitly instead of relying on the runner image.
- **Documentation corrected** — the false claim that Windows/macOS binaries were "pre-built and unaffected" was removed everywhere; crates.io ships no pre-built binaries.
- **No runtime changes** — same flags, same JSON schema, same dependencies as v0.7.3.

## What's new in v0.7.5 (2026-06-14)

- **GAP-WS-29/30/31 closed (build experience, Windows).** Extended the v0.7.4 NASM preflight to also detect CMake (crate cmake 0.1.58 needs `cmake.exe` in PATH BEFORE `enable_language(ASM_NASM)` is evaluated), MSVC compiler and linker (`cl.exe`/`link.exe` — need `Launch-VsDevShell.ps1` to be on PATH), and Perl (`perl.exe` for BoringSSL's perlasm generator). New `build.rs` preflight aborts in seconds with the exact fix for each of the four tools. Escape hatches: `DDG_SKIP_NASM_CHECK=1`, `DDG_SKIP_CMAKE_CHECK=1`, `DDG_SKIP_MSVC_CHECK=1`, `DDG_SKIP_PERL_CHECK=1`. Root cause: the C++ CMake tools for Windows sub-component of the Visual Studio Installer is deselected by default — installing only the C++ workload does NOT provide CMake.
- **Extended helper `scripts/install-windows.ps1`** — now detects and auto-installs CMake (`winget install -e --id Kitware.Cmake` or choco), Perl (`winget install -e --id StrawberryPerl.StrawberryPerl`), and reports the exact MSVC install/Launch-VsDevShell.ps1 instruction (MSVC is too large to auto-install). New `--check-only` mode produces a tabular report suitable for CI gates and human support.
- **New `scripts/check-windows-toolchain.ps1`** — standalone diagnostic (no installs) that checks all 7 tools (cargo, rustc, cmake, nasm, cl.exe, link.exe, perl) and emits text or JSON output. Exit code 0 if all present, 1 otherwise. Use for support tickets and CI gates.
- **New `docs/INSTALL-WINDOWS.md`** — step-by-step guide covering 5 installation methods (Visual Studio Installer + standalone tools; all-standalone via winget; Chocolatey only; helper script; standalone diagnostic). Includes troubleshooting for each of the 4 GAPs and the `DDG_SKIP_*_CHECK` escape hatches.
- **Documentation corrected** — the false claim that "VS Build Tools with C++ workload provides CMake" was replaced in `CROSS_PLATFORM.md`, `README.md`, `README.pt-BR.md`, `skill/duckduckgo-search-cli-en/SKILL.md`, `skill/duckduckgo-search-cli-pt/SKILL.md`, `llms-full.txt` and `llms.pt-BR.txt`. The C++ workload does NOT include the C++ CMake tools sub-component — it must be selected manually in the Visual Studio Installer.
- **No runtime changes** — same flags, same JSON schema, same dependencies as v0.7.4. crates.io still ships NO pre-built binaries for any platform.

## What's new in v0.7.3

- **GAP-WS-27 fixed (CRITICAL).** The macOS CAPTCHA interstitial that returned HTTP 200 with `quantidade_resultados: 0` while Windows returned full results is closed. Root cause: `rustls` produces a TLS fingerprint recognisable by Cloudflare Bot Management (JA4_o vector). v0.7.3 replaces `reqwest 0.12` + `rustls-tls` with `wreq 6.0.0-rc.29` + BoringSSL (`boring2` 4.15.11) + `wreq-util 3.0.0-rc.12`. BoringSSL produces a JA4_o fingerprint identical to Chrome/Safari. Reproduction: same query, same machine — v0.7.2 returned 0 results in 1695ms; v0.7.3 returns 5 results in 735ms. See `docs/decisions/0001-tls-boring-via-wreq.md`.
- **New `session` feature (cookie persistence + warm-up).** New flags: `--no-warmup` (skip `GET https://duckduckgo.com/` warm-up), `--no-cookie-persistence` (in-memory only), `--cookies-path <PATH>` (override XDG default). Cookie jar is written to `~/.config/duckduckgo-search-cli/cookies.json` (Linux), `%APPDATA%\duckduckgo-search-cli\cookies.json` (Windows), or `~/Library/Application Support/duckduckgo-search-cli/cookies.json` (macOS) with Unix permissions `0o600`.
- **New `probe-deep` feature (CAPTCHA interstitial detection).** New flags: `--probe-deep` (run a real search query and classify the body as `ok` or `captcha` based on Cloudflare and DuckDuckGo markers), `--allow-lite-fallback` (opt-in to automatic `html → lite` fallback when CAPTCHA is detected).
- **Build prerequisites added.** Compiling from source now requires `cmake`, `perl`, `pkg-config`, and `libclang-dev` on Linux. crates.io ships NO pre-built binaries — `cargo install` always compiles from source; on native Windows MSVC the NASM assembler is also required (GAP-WS-28, preflight added in v0.7.4). CI matrix in `.github/workflows/release.yml` installs these packages in the Linux jobs.
- **New modules** — `session_warmup` (XDG path resolution), `wreq_cookie_adapter` (JSON ↔ `wreq::cookie::Jar` bridge), `probe_deep` (interstitial detection).
- **Zero breaking changes to JSON output schema** — all v0.7.2 fields remain present.
- **Test count: 292 lib** (was 279 in v0.7.2) + 18 wiremock + other integrations. 0 clippy warnings, 0 fmt diff, 2 cargo-deny warnings (RUSTSEC-2025-0057 and RUSTSEC-2025-0052, both already in the ignore list).
- **Binary size +20 MB** (BoringSSL is statically linked). Release build time +30s to +90s.


## What's new in v0.7.2

- **Security advisory fix (RUSTSEC-2026-0009)**: `time 0.3.40` denial-of-service via RFC 2822 stack exhaustion was being pulled in transitively via `cookie_store 0.22.0` → `reqwest 0.12.28`. v0.7.2 pins `time = "0.3.47"` as a direct dep to override the transitive constraint.
- **`rand` 0.10 migration**: dev-deps (proptest 1.11+, getrandom 0.4+) unified on rand 0.10. Convenience methods moved from `Rng` to `RngExt`. All internal call sites updated.
- **MSRV bump**: `rust-version` raised from 1.75 to 1.88 (required by `time 0.3.47+` and `rand 0.10`).
- **402 tests passing** (289 lib + 101 integration + 12 doctest), 0 failures.

## What's new in v0.7.1

- **`rand` 0.9 migration** (zero breaking changes). `random_range`, `random_bool`, `random`, and `IndexedRandom::choose`.
- **MSRV bump** to 1.85.
- **reqwest builder cleanup**: `gzip` and `brotli` features removed (now handled via `Accept-Encoding`).
- **392 tests passing**.

## What's new in v0.7.0

- **New subcommand `deep-research`** — query fan-out pipeline for multi-hop LLM research
  - Splits one query into 1..=12 sub-queries via five canonical heuristic templates (aspect, comparison, timeline, opinion, cause) or a manual file
  - Fans out in parallel via the existing `JoinSet` + `Semaphore` executor
  - Aggregates with Reciprocal Rank Fusion (K=60) or canonical-URL deduplication
  - Optional synthesis to Markdown, PlainText, or JSON with numbered references and a token budget
- **Four new public modules** — `deep_research`, `decomposition`, `aggregation`, `synthesis` — composable from downstream crates
- **New dependencies** — `url = "2"` (canonicalisation), `regex = "1"` (composite-query detection), `proptest = "1"` (dev, property tests)
- **Zero breaking changes** — `buscar`, `init-config`, default-config JSON schema, and all exit codes are byte-for-byte identical to v0.6.x
- **392 tests passing** (279 lib + 12 doc + 101 integration), 8 validation gates OK


## What's new in v0.6.5

- **CRITICAL FIX (MP-26)**: Windows build compiles again. `windows-sys 0.59+` changed `HANDLE` from `isize` to `*mut c_void`. Replaced `handle != 0 && handle != usize::MAX` with `!handle.is_null() && handle != INVALID_HANDLE_VALUE`.
- **CI fix (CI-01)**: 6 latent clippy errors in v0.6.4 broke the `validate` job on all 3 SOs. Fixed: `doc_markdown` on 3 strings (`PowerShell`, `rules_rust.md`, `TempDir`), `needless_return`, `missing_debug_implementations` on `ChromeBrowser` and `CircuitBreakerMap`.
- **Preventive FFI lints**: `improper_ctypes = "deny"` and `improper_ctypes_definitions = "deny"` in `Cargo.toml`.
- **WS-11**: 5 property tests (invariants) in `extraction.rs` — no new dependency.
- **WS-12**: Per-host circuit breaker in `content_fetch.rs` — 3 consecutive failures open the circuit for 30s.
- **WS-23**: WireMock test validates the `Retry-After` header on 429 responses.
- **WS-25**: `indicatif = "0.18"` added for the ProgressBar in long crawls.
- **405 tests passing** (current project total at v0.7.5 — historical v0.6.5 release had 333), 8 validation gates OK (fmt, clippy, test, doc, audit, deny, publish dry-run, package list).


## What's new in v0.6.4

- Adaptive anti-bot identity pool (12 identities, 4 browser families × 3 platforms)
- 5-level cascade rotation when DDG blocks (HTTP 202/403/429)
- New JSON fields: `metadados.identidade_usada` and `metadados.nivel_cascata` (additive, non-breaking)
- New CLI flags: `--probe` (pre-flight health check) and `--identity-profile` (pin profile)
- `--seed` flag now also controls identity pool rotation (previously UA only)


## Canonical Documentation

- `README.md` — overview, flags, exit codes, bilingual examples
- `docs/HOW_TO_USE.md` — pedagogical step-by-step guide
- `docs/COOKBOOK.md` — 23 ready-to-use recipes for search, ETL, monitoring, and deep research
- `docs/INTEGRATIONS.md` — integrations with 16+ AI agents (Claude, GPT, Gemini, Cursor…)
- `docs/AGENTS-GUIDE.md` — stdin/stdout contract and full JSON schema reference
- `docs/CROSS_PLATFORM.md` — setup on Linux, macOS, Windows, and Docker
- `docs/AGENTS.md` — quick integration guide for AI agents
- `docs/AGENT_RULES.md` — 30+ MUST/NEVER rules for production use by agents
- `docs/MIGRATION.md` — migration guide between versions
- `docs/TESTING.md` — test execution and categorization
- `skill/duckduckgo-search-cli-en/SKILL.md` — Claude/OpenCode skill for agents
- `skill/duckduckgo-search-cli-pt/SKILL.md` — Portuguese skill for agents
- `CHANGELOG.md` — version history (EN)
- `CHANGELOG.pt-BR.md` — version history (PT)
- `INTEGRATIONS.md` — pointer to full integration catalog
- `CONTRIBUTING.md` — contributor onboarding
- `CODE_OF_CONDUCT.md` — community standards
- `SECURITY.md` — vulnerability disclosure policy


## Quick Start

### Basic search (plain text on TTY)

```bash
timeout 30 duckduckgo-search-cli "rust async runtime"
```

### Machine-readable JSON (piped)

```bash
timeout 30 duckduckgo-search-cli -q -f json --num 10 "rust async runtime" \
  | jaq -r '.resultados[].url'
```

### Pre-flight health check (v0.6.4+, preserved in v0.6.5)

```bash
timeout 15 duckduckgo-search-cli --probe
# {"endpoint":"html","has_set_cookie":false,"latency_ms":109,"status":200,"type":"probe","url":"https://html.duckduckgo.com/html/"}
```

### Batch mode (multi-query)

```bash
printf 'query one\nquery two\nquery three\n' > /tmp/q.txt
timeout 300 duckduckgo-search-cli \
  --queries-file /tmp/q.txt \
  -q -f json --parallel 3 --per-host-limit 1 --retries 3 \
  --global-timeout 280 > /tmp/multi.json
```

### Content extraction (long crawls, WS-25 ProgressBar)

```bash
timeout 120 duckduckgo-search-cli -q -f json --num 5 \
  --fetch-content --max-content-length 5000 \
  "rust embedded systems" | jaq '.resultados[].url'
```


## Canonical Invocations

```bash
# Single query
duckduckgo-search-cli -q -f json "query" | jaq '.resultados | length'

# Multi-query (use --queries-file, NEVER shell loops)
duckduckgo-search-cli -q -f json "one" "two" | jaq '.buscas[0].resultados | length'

# Filter by URL
duckduckgo-search-cli -q -f json "rust" | jaq -r '.resultados[].url'

# Detect identity used (v0.6.4+)
duckduckgo-search-cli -q -f json "rust" | jaq -r '.metadados.identidade_usada // "n/a"'

# Detect cascade level (v0.6.4+)
duckduckgo-search-cli -q -f json "rust" | jaq '.metadados.nivel_cascata // 0'

# Deep research (v0.7.0+): default heuristic, RRF, no synthesis
timeout 60 duckduckgo-search-cli -q -f json deep-research "rust http client 2026" \
  | jaq '.resultados[] | {titulo, url, score}'

# Deep research with synthesis (Markdown report)
timeout 120 duckduckgo-search-cli -q -f json deep-research "tokio vs async-std 2026" \
  --synthesize --budget-tokens 1500 --synth-format markdown \
  --fetch-content --max-content-length 6000 | jaq '.sintese'

# Deep research with manual sub-queries from a file
cat > /tmp/qs.txt <<EOF
# Visão geral
what is tokio runtime 2026
# Comparação
tokio vs async-std vs smol
EOF
timeout 60 duckduckgo-search-cli -q -f json deep-research "tokio runtime 2026" \
  --sub-queries-file /tmp/qs.txt --aggregate dedupe-by-url \
  | jaq '.metadados.sub_queries | length'
```


## Exit Codes

| Code | Meaning           | Agent action                              |
|------|-------------------|-------------------------------------------|
| `0`  | Success           | Parse `.resultados`                       |
| `1`  | Runtime error     | Read stderr; retry once with `-v`         |
| `2`  | Config error      | Re-run `init-config --force`              |
| `3`  | Anti-bot block    | Back off 300+ s; switch `--endpoint lite` |
| `4`  | Global timeout    | Raise `--global-timeout`; reduce `--parallel` |
| `5`  | Zero results      | Refine query or change `--lang`           |


## Anti-Patterns to Avoid

- NEVER parse `text` or `markdown` output with machines (use `json` + `jaq`)
- NEVER omit `timeout` (pipelines hang indefinitely)
- NEVER pass proxy credentials in argv (use `HTTPS_PROXY` env var)
- NEVER raise `--parallel` above 5 (triggers anti-bot)
- NEVER use `--stream` (placeholder, not implemented)
- NEVER inject custom `Sec-Fetch-*` or `Accept-Language` headers


## License

Dual-licensed under MIT OR Apache-2.0. See `LICENSE` for details.
