Expand description
Streaming multipart/mixed parser for POST /scrape/batch.
The API emits one part per scrape as each completes. This module
reads the response body as a stream of Bytes chunks and yields
(headers, body) per part as they arrive.
Zero new dependencies — only reqwest, bytes, and futures-util
that are already in Cargo.toml.
Structs§
- Options for
Client::scrape_batch_with_options. - One multipart part: header map (lowercased keys) plus body bytes.
- Stream adapter: wraps a reqwest bytes stream and yields one
BatchPartper multipart section as the body arrives. - A proxified batch part surfaced as a native Response-like value. The part body is the raw upstream response (HTML, JSON, binary, etc.) — not a JSON envelope.
reqwest::Responseis tied to a live connection so we cannot re-synthesize one from bytes; this struct carries the same fields a caller needs.
Enums§
- Wire format for the per-part body. JSON is the default; Msgpack matches the Scrapfly API’s msgpack negotiation.
- Per-part outcome yielded by
Client::scrape_batch.
Functions§
- Synthesize a
BatchProxifiedResponsefrom a proxified batch part. Restores the upstream HTTP status fromX-Scrapfly-Scrape-Status, merges upstream headers (after stripping theX-Scrapfly-Upstream-prefix) with Scrapfly metadata headers, and exposes the raw body. - Decode a part body according to its Content-Type. Supports
application/json(default) andapplication/msgpack. - Convenience: take a reqwest
Responsewhose Content-Type is multipart/mixed and return a typedStream<Item=BatchPart>.