nmaprs — Engineering Report
Description: nmaprs — engineering report. Pipelined raw I/O, 16-shard concurrency per family, MatchPoints OS detection, nmap-service-probes regex matcher, parity matrix vs upstream nmap, performance + architecture deep dive.

## >_EXECUTIVE SUMMARY

nmaprs is a from-scratch Rust implementation of the `nmap` CLI dialect, with packet-level evasion, pipelined raw I/O on IPv4 + IPv6, and OS / version detection scored against nmap's own reference databases (`nmap-os-db`, `nmap-service-probes`). The CLI surface accepts the union of `nmap --help` and the long-option set from upstream `nmap.cc` — existing nmap scripts, integrations, and XML pipelines drop in unchanged.

What it does ship that vanilla nmap doesn't: async / parallel by default (no `-T5` needed for "fast"), SOCKS4 + HTTP CONNECT proxy support for TCP connect scans, custom DNS resolution via `hickory-resolver`, sharded raw send/recv pipelines that scale to 16 concurrent shards per address family with race-free reply matching, and `--resume` JSON checkpoints across all scan types (TCP connect, UDP, raw half-open, IP-protocol, SCTP, idle, FTP bounce). What it doesn't ship: the NSE Lua runtime.

128
CLI options
16
Concurrent shards / family
11
Scan techniques
9
Discovery probes (-P*)
5
Output formats (nmap-compatible)
2
Binaries (nmaprs + nms)
1,028
man nmaprsall lines

## ~SCAN ENGINE ARCHITECTURE

Per address family, raw scans spin up a dedicated recv thread blocking on `poll(2)`, a sharded send pool (up to 16 concurrent shards), and a global rate limiter shared across IPv4 + IPv6. The recv-key registration happens before each send to eliminate the classic "reply landed before we knew to listen" race that ad-hoc raw scanners hit at high concurrency.

Layer | Implementation
--- | ---
Raw socket I/O | `pnet` + `socket2` — IPv4 layer-3 datagrams, IPv6 raw via `IPPROTO_RAW` + `IPV6_HDRINCL` on Unix
Async runtime | `tokio` multi-thread — one reactor pool, raw recv lives on dedicated thread (not the reactor)
Mixed IPv4 + IPv6 dispatch | `tokio::join` — both engines concurrent; per-`IpAddr` host-timeout clock shared
Sharding | Up to 16 concurrent pipelines per family, bounded by `effective_probe_concurrency()` from timing template / `--max-parallelism`
Recv key registration | Registered before send via `DashMap`; reply dispatcher drops to per-target oneshot channel; no lost replies under load
UDP ICMP unreach listener | One `poll(2)` + burst-recv thread per scan (not per host batch); IPv4 type-3 + ICMPv6 type-1 classification
SCTP CRC32c | In-tree CRC32c for SCTP common header + chunks; INIT-ACK / COOKIE-ACK → `open`, ABORT → `closed`
Service / version | `nmap-service-probes` parser; `match` / `softmatch` via Rust `regex` (Perl-only features skipped); TLS probes via `rustls` when port matches `sslports`
OS detection | Subject fingerprint (SEQ / OPS / WIN subset, extras in progress); scored against `nmap-os-db` with MatchPoints + expr_match (same algorithm nmap uses); falls back to ICMP TTL heuristic on minimal data
DNS | `hickory-resolver` with `--dns-servers` overrides, async, parallel; `--system-dns` escape hatch
Proxy | `tokio-socks` for SOCKS4; in-tree HTTP CONNECT; both apply to TCP connect scans
Rate limiting | One global token-bucket limiter on probe starts; IPv4 + IPv6 share; respects `--min-rate` floor by inferring parallelism upward when needed
Checkpoint format | JSON of completed `(host, port)` pairs; `--resume` filters per batch and merges at end; works with `--min-hostgroup` / `--max-hostgroup`

## $PARITY MATRIX vs UPSTREAM NMAP

Every nmap flag that nmaprs actually parses (128 options) plus where it stops short of nmap's behavior. Numbers from the project's TRUTH TABLE in the README.

Area | nmaprs status
--- | ---
TCP connect (`-sT`, default unprivileged) | Implemented — async, parallel, timeout-bound
TCP SYN raw (`-sS`, IPv4 + IPv6) | Implemented — pipelined recv, 16-shard send per family, connect fallback on raw failure
NULL / FIN / Xmas / Maimon (`-sN`/`-sF`/`-sX`/`-sM`) | Implemented — same raw sharded pipeline; Maimon sends FIN+ACK
ACK (`-sA`) / Window (`-sW`) | Implemented — firewall mapping (RST → `unfiltered`) and window-state classification
UDP (`-sU`) | Implemented — ICMP unreach listener distinguishes port-unreach (`closed`) from other unreach (`filtered`)
SCTP (`-sY` INIT / `-sZ` COOKIE-ECHO) | Implemented — CRC32c, IPv4 + IPv6, mixed-family `tokio::join`
Idle scan (`-sI zombie[:probeport]`) | Implemented — IPv4 only, sequential IP-ID sampling; IPv6 skipped (warning)
FTP bounce (`-b user:pass@host:port`) | Implemented — IPv4 only (`PORT` command); 150/125/250 → `open`, 425/426/421 → `closed`
IP protocol (`-sO`) | Implemented — IPv4 + IPv6 on Unix; `-p` is protocol numbers 0..=255; `-F` uses `nmap-protocols`
Discovery (`-PS`/`-PA`/`-PU`/`-PY`/`-PE`/`-PP`/`-PM`/`-PO`/`-PR`) | Implemented — ARP auto on local IPv4 subnets (`--disable-arp-ping` to opt out)
Ping scan (`-sn`) | Implemented — raw ICMP via `pnet`, falls back to system `ping`; output formats write host-up lines
IPv6 (`-6`) | Implemented — targets + scans including raw SYN when privileged
Traceroute (`--traceroute`) | Implemented — system `traceroute`/`tracert`; up to `min(parallelism, 32)` concurrent
`--iL` / `--iR` / `--exclude` / `--excludefile` | Implemented — parallel resolution preserving order
`--resume` (JSON checkpoint) | Implemented — TCP connect, UDP, raw half-open TCP, IP protocol scan
Evasion (`-g`, `--ttl`, `--badsum`, `-D`, `-S`, `--data*`, `-f`/`--mtu`, `--spoof-mac`) | Implemented — wired at the packet level for raw TCP scans
`--proxies` / `--proxy` | Implemented — SOCKS4 + HTTP CONNECT for TCP connect scans
`--dns-servers` | Implemented — hickory-resolver, custom resolver IPs comma-separated
Auto privilege detect (`geteuid`) | Implemented — downgrade SYN→connect when unprivileged; `--privileged` forces raw
Output: `-oN` / `-oG` / `-oX` / `-oA` / `-oS` | Implemented — `<nmaprun>` root with nmap-compatible elements; existing parsers ingest it
Timing template (`-T0`..`-T5`) | Implemented — parallelism + RTT bounds inferred from template; explicit flags override
`--min-rate` / `--max-rate` / parallelism / hostgroup / RTT / retries / scan-delay / `--stats-every` | Implemented — one global rate limiter across families
`--scanflags` (custom TCP flags) | Implemented — SYN/ACK/FIN/RST/PSH/URG/ECE/CWR; raw TCP scans only
Port specs (`-p`, `-F`, `--top-ports`, `--port-ratio`) | Implemented — embedded TCP frequency list; `-p -` for all 65536
OS detection (`-O`) | Partial — IPv4 raw + MatchPoints scoring against `nmap-os-db` when DB available + open/closed TCP found. IPv6 = TTL heuristic only
Service / version (`-sV`) | Partial — full `nmap-service-probes` support except Perl-only regex features
Scripts (`--script` / `-sC`) | Partial — `default` + `banner` builtins; no Lua engine
`-oM` / `-oH` | Partial — `-oM` = grepable; `-oH` placeholder
`--osscan-limit` / `--osscan-guess` / `--max-os-tries` | Partial — limits + DB example titles; no multi-round retry
`--stylesheet` / `--webxml` / `--no-stylesheet` | Partial — XML preamble + `xml-stylesheet` PI
NSE Lua runtime | Not implemented — use upstream nmap when Lua is required

## &SUBSYSTEM DEEP DIVES

### // PIPELINED RAW SEND/RECV (16 SHARDS / FAMILY)

The naive raw scanner pattern — `send packet; recv loop; classify` — loses replies under load when the reply arrives before the recv loop registers the per-target callback. nmaprs's raw engine inverts the order:

1. Register the recv key first in a `DashMap<(IpAddr, port, scan-type), oneshot::Sender>`.
2. Then send the probe from the main thread of the shard.
3. The dedicated recv thread demuxes incoming frames against the registered keys and fires the matching oneshot.
4. Timeout falls back to `filtered` after the per-probe RTT cap.
5. Connect-mode fallback per address family kicks in when raw send fails outright (PERM, EAFNOSUPPORT, etc.) — not when probes individually time out.

Shard count defaults to `min(16, effective_probe_concurrency())` per family. `effective_probe_concurrency()` is the bound between the timing template's parallelism estimate and explicit `--min-parallelism` / `--max-parallelism` / `--min-rate` floors.

### // nmap-os-db SCORING

When `-O` is on, IPv4 is privileged, the scan found both open and closed TCP ports, and `--datadir` contains `nmap-os-db`:

1. Raw Layer-3 probes build a subject fingerprint — SEQ, OPS, WIN subsets implemented today; the full nmap probe set (T1..T7, ECN, U1, IE1, etc.) is in progress.
2. Each candidate `nmap-os-db` entry is scored via MatchPoints + expr_match — the same algorithm upstream nmap uses.
3. Top-scoring matches print with their `OS:` / `Class:` / `CPE:` values.
4. When raw probes or matching data are unavailable, falls back to ICMP TTL heuristic + example `Class:` titles from the legacy DB parse.

IPv6 OS detection currently uses only the TTL heuristic.

### // nmap-service-probes MATCHER

The `-sV` path parses `nmap-service-probes` and runs probes parallel-per-open-port:

- TCP + UDP probe blocks — both directions.
- ports / sslports filtering — ports outside the probe's set are skipped.
- TLS via `rustls` when the port matches the probe's `sslports`.
- rarity vs `--version-intensity` — high-rarity probes are skipped at low intensity, identical to nmap's behavior.
- match / softmatch with full template fields — `p/` (product), `v/` (version), `i/` (info), `o/` (OS), `d/` (device type), `cpe:/` (CPE URIs).
- Perl-only regex features in the upstream patterns are detected and the matcher skips that entry — this is the source of the "partial" parity rating.

### // EVASION AT PACKET LEVEL

Source-port spoofing, custom TTL, bad checksums, decoys, source-IP spoofing, custom payload (hex / string / random length), fragmentation (`-f`, `--mtu`), and MAC spoofing on ARP discovery frames are all wired into the raw IPv4 packet construction for `-sS` / `-sN` / `-sF` / `-sX` / `-sM` / `-sA` / `-sW`. These are not wrappers around an external `scapy`-style helper — they manipulate the packet bytes nmaprs writes to the raw socket.

## !WHEN TO USE NMAP INSTEAD

If your workflow requires authoritative behavior of any of the items below, reach for upstream `nmap`:

- NSE Lua scripts — nmaprs ships `banner` + `default` Rust builtins, not the Lua runtime. Any script that ships with nmap (vuln checks, HTTP enum, SNMP, SMB, etc.) needs upstream.
- Full IPv6 OS detection — nmaprs's IPv6 OS detection is a TTL heuristic, not the IPv6 OS probe engine nmap has.
- Multi-round OS retry / `--max-os-tries` with adaptive intervals — nmaprs honors the flag but doesn't do the full retry behavior.
- The exact `-oH` hex dump format — currently a placeholder file in nmaprs.
- Service-probe entries that use Perl-only regex features — nmaprs's `regex` matcher skips them with a warning.

## /SECURITY & ETHICS

Run nmaprs only against networks and hosts you own or have explicit written permission to scan. Unauthorized scanning is illegal in many jurisdictions. The author and contributors disclaim responsibility for misuse. This tool exists because Rust's memory safety + async runtime + raw-socket bindings combine to make a credible parallel scanner without C's footguns — not to make scanning easier for hostile actors.

## #PROJECT METADATA

Item | Value
--- | ---
License | MIT
Author | MenkeTechnologies
Repository | github.com/MenkeTechnologies/nmaprs
Crate | crates.io/crates/nmaprs
API docs | docs.rs/nmaprs
Man pages | `man nmaprs` (506 lines, short ref) + `man nmaprsall` (1028 lines, full ref)
Issues | github.com/MenkeTechnologies/nmaprs/issues