scrapfly_sdk

Module batch

Source
Expand description

Streaming multipart/mixed parser for POST /scrape/batch.

The API emits one part per scrape as each completes. This module reads the response body as a stream of Bytes chunks and yields (headers, body) per part as they arrive.

Zero new dependencies — only reqwest, bytes, and futures-util that are already in Cargo.toml.

Structs§

  • Options for Client::scrape_batch_with_options.
  • One multipart part: header map (lowercased keys) plus body bytes.
  • Stream adapter: wraps a reqwest bytes stream and yields one BatchPart per multipart section as the body arrives.
  • A proxified batch part surfaced as a native Response-like value. The part body is the raw upstream response (HTML, JSON, binary, etc.) — not a JSON envelope. reqwest::Response is tied to a live connection so we cannot re-synthesize one from bytes; this struct carries the same fields a caller needs.

Enums§

  • Wire format for the per-part body. JSON is the default; Msgpack matches the Scrapfly API’s msgpack negotiation.
  • Per-part outcome yielded by Client::scrape_batch.

Functions§

  • Synthesize a BatchProxifiedResponse from a proxified batch part. Restores the upstream HTTP status from X-Scrapfly-Scrape-Status, merges upstream headers (after stripping the X-Scrapfly-Upstream- prefix) with Scrapfly metadata headers, and exposes the raw body.
  • Decode a part body according to its Content-Type. Supports application/json (default) and application/msgpack.
  • Convenience: take a reqwest Response whose Content-Type is multipart/mixed and return a typed Stream<Item=BatchPart>.