# hf-fetch-model v0.9.0 Roadmap — Safetensors Remote Inspection & Disk Usage

**Date:** March 23, 2026
**Status:** Proposed
**Supersedes:** v0.9.0-roadmap.md V1 (March 21, 2026) — cache management scope moved to [v0.10.0](v0.10.0-roadmap.md)
**Context:** v0.8.x delivered cache-awareness (`list-files --show-cached`, `--dry-run` cache status, partial detection). Meanwhile, the **anamnesis** crate (tensor format transformation — FP8, GPTQ, AWQ dequantization) now needs to read safetensors tensor metadata *before* downloading multi-GiB files. hf-fetch-model already has the HTTP Range infrastructure (chunked downloads) and the HF API integration — it is the natural place for remote header fetching. v0.9.0 adds **safetensors remote header inspection** (the primary feature) and **`du`** (disk usage, nearly zero new code).

---

## Goals

1. **Safetensors header inspection** — read tensor metadata (names, shapes, dtypes, offsets) from local cache or remote HF repos (via two HTTP Range requests), without downloading full files
2. **Disk usage (`du`)** — measure what's taking space in the local HF cache

---

## Motivation: anamnesis Integration

anamnesis provides quantization scheme detection and dequantization. Its `parse_safetensors_header()` function (in `anamnesis::parse::safetensors`) classifies tensors by role (`Quantized`, `Scale`, `Passthrough`, `ZeroPoint`, `GroupIndex`) and detects the quantization scheme (`FineGrainedFp8`, `PerChannelFp8`, `PerTensorFp8`, `Gptq`, `Awq`, `Unquantized`).

Today, anamnesis requires the **entire file on disk** to parse a header. But the safetensors format stores the JSON header in the first bytes of the file — two Range requests (8 bytes for the header length, then the header JSON) are enough to read it remotely. This means hf-fetch-model can fetch the raw header bytes from HuggingFace and anamnesis can interpret them, enabling workflows like:

- **Pre-download inspection:** "Is this model FP8 or GPTQ? What dtypes are the tensors?"
- **Selective download:** "Only download the model if it's a quantization scheme anamnesis supports"
- **Metadata cataloging:** "List all tensor names and shapes across a repo's safetensors files"

### Responsibility boundary

| Crate | Responsibility |
|-------|---------------|
| **hf-fetch-model** | Fetch raw header bytes (HTTP Range or local file), parse the safetensors JSON into tensor names/shapes/dtypes/offsets. No quantization-specific logic. |
| **anamnesis** | Interpret tensor metadata: classify roles, detect quantization scheme, find scale companions. Consumes hf-fetch-model's `TensorInfo` to build its own `TensorEntry` with `TensorRole`. |

hf-fetch-model does **not** depend on anamnesis. The dependency flows one way: anamnesis (or any consumer) calls hf-fetch-model's `inspect_safetensors()` and maps the result into its own types.

---

## Existing Infrastructure

| Module | What exists | What v0.9.0 adds |
|--------|------------|-------------------|
| `repo.rs` | `list_repo_files_with_metadata()` — direct HTTP GET to HF API with `?blobs=true`, returns `Vec<RepoFile>` with filename, size, SHA256 | `inspect` reuses this to discover `*.safetensors` files when no filename is specified |
| `cache.rs` | `hf_cache_dir()`, `cache_summary()` → `Vec<CachedModelSummary>`, `read_ref()`, `count_snapshot_files()` | `du` reuses `cache_summary()` directly; `inspect` (cache-first by default) resolves local paths via `hf_cache_dir()` + `read_ref()` |
| `config.rs` | `FetchConfig` with `token`, `revision` | `inspect` accepts the same `--token` and `--revision` flags |
| `error.rs` | `FetchError::Http`, `FetchError::Io` | New `FetchError::SafetensorsHeader` variant for malformed headers |
| `bin/main.rs` | `format_size()`, table display patterns from `list-files` and `status` | `inspect` and `du` follow the same display conventions |

**New dependency:** None. The safetensors header is plain JSON (parsed with `serde_json`, already present). The 8-byte length prefix is little-endian `u64` (parsed with `u64::from_le_bytes`). HTTP Range requests use `reqwest` (already present).

---

## Feature 1: Safetensors Remote Header Inspection

### Why

Safetensors files encode a JSON header in their first bytes: an 8-byte little-endian length prefix followed by the JSON blob. The header contains every tensor's name, dtype, shape, and byte offsets — everything needed to understand a model's structure without downloading GiBs of weight data. Two HTTP Range requests suffice:

1. `Range: bytes=0-7` → 8-byte `u64` header length (little-endian)
2. `Range: bytes=8-{8+length-1}` → JSON header

### CLI

```
hf-fm inspect <REPO_ID> [FILENAME] [--revision <REV>] [--token <TOKEN>] [--cached] [--no-metadata] [--json]
```

| Flag | Effect |
|------|--------|
| `FILENAME` | Specific `.safetensors` file to inspect. If omitted, inspects all `*.safetensors` files in the repo. |
| `--revision` | Target branch, tag, or commit (default: `main`). |
| `--token` | HuggingFace API token (or `HF_TOKEN` env var). |
| `--cached` | Cache-only mode: read from local cache, fail if the file is not cached. Without this flag, inspect still **checks the cache first** and only makes HTTP Range requests if the file is not found locally. |
| `--no-metadata` | Suppress the `Metadata:` line in human-readable output. By default, `__metadata__` is printed when present (see below). Ignored with `--json`. |
| `--json` | Output the full `SafetensorsHeaderInfo` as JSON instead of the human-readable table. For programmatic consumption (e.g., anamnesis piping `hf-fm` output). |

**Default behavior (no `--cached`):** try local cache first, fall back to HTTP Range requests. This means `hf-fm inspect` is free and instant when the model is already downloaded — no network round-trip at all. The `--cached` flag is for when you want to guarantee no network access (e.g., offline environments, CI).

### Output: single file (human-readable)

```
$ hf-fm inspect deepseek-ai/DeepSeek-V3 model-00001-of-000163.safetensors
  Repo:     deepseek-ai/DeepSeek-V3
  File:     model-00001-of-000163.safetensors
  Source:   cached
  Header:   53.2 KiB (JSON), 4.85 GiB total
  Metadata: format=pt

  Tensor                                            Dtype    Shape            Size       Params
  model.embed_tokens.weight                         BF16     [129280, 7168]   1.73 GiB   927.0M
  model.layers.0.input_layernorm.weight             BF16     [7168]           14.0 KiB   7.2K
  model.layers.0.self_attn.q_proj.weight            F8E4M3   [7168, 7168]    48.9 MiB   51.4M
  model.layers.0.self_attn.q_proj.weight_scale_inv  BF16     [56, 56]        6.1 KiB    3.1K
  ...
  ──────────────────────────────────────────────────────────────────────────────────────────────
  142 tensors, 1.02B params
```

The **Size** column (from `data_offsets`) next to **Params** reveals quantization at a glance: 51.4M params at 48.9 MiB = 1 byte/param (FP8). If it were BF16, the same tensor would be 97.8 MiB.

For an AWQ model, the compact `Metadata:` line surfaces the quantization config directly:

```
$ hf-fm inspect casperhansen/llama-3.2-1b-instruct-awq model.safetensors
  Repo:     casperhansen/llama-3.2-1b-instruct-awq
  File:     model.safetensors
  Source:   remote (2 HTTP requests)
  Header:   8.7 KiB (JSON), 823 MiB total
  Metadata: quant_method=awq, bits=4, group_size=128, version=gemm

  Tensor                                            Dtype  Shape            Size       Params
  ...
```

The `Metadata:` line is printed by default when `__metadata__` is present — it's typically 2-5 key=value pairs on one line. Use `--no-metadata` to suppress it. This is critical for anamnesis: the metadata entries (especially `quant_method`, `bits`, `group_size`) are how anamnesis distinguishes GPTQ from AWQ without downloading weights. For files with many metadata entries (>5), the line wraps at terminal width.

### Output: single file (JSON)

```
$ hf-fm inspect deepseek-ai/DeepSeek-V3 model-00001-of-000163.safetensors --json
{
  "tensors": [
    {
      "name": "model.embed_tokens.weight",
      "dtype": "BF16",
      "shape": [129280, 7168],
      "data_offsets": [0, 1854668800]
    },
    ...
  ],
  "metadata": { "format": "pt" },
  "header_size": 54477,
  "file_size": 5204836352
}
```

The JSON output mirrors `SafetensorsHeaderInfo` directly — consumers like anamnesis can deserialize it into their own types without parsing column-aligned text. This enables workflows like `hf-fm inspect ... --json | amn classify`.

### Output: all safetensors in repo

For sharded models with an index file, the summary is built from the index (1 request) rather than fetching each shard's header individually:

```
$ hf-fm inspect deepseek-ai/DeepSeek-V3
  Repo:   deepseek-ai/DeepSeek-V3
  Source: shard index (model.safetensors.index.json)

  File                                    Tensors
  model-00001-of-000163.safetensors       142
  model-00002-of-000163.safetensors       128
  ...
  ─────────────────────────────────────────────────────────────────
  163 shards, 18,432 tensors
  Hint: use `hf-fm inspect deepseek-ai/DeepSeek-V3 <filename>` for per-tensor detail
```

Note: the shard index maps tensor names to shard filenames but does not contain shapes, dtypes, or offsets. The summary table shows tensor count per shard (derived from `weight_map`) but no Params or Size columns. To get full per-tensor detail, specify the shard filename — the single-file path uses Range requests for the complete header.

### Implementation

#### Resolution order (cache-first)

```
1. Try local cache:
   a. hf_cache_dir() → ~/.cache/huggingface/hub/
   b. models--{org}--{name}/refs/{revision} → commit hash (via read_ref())
   c. models--{org}--{name}/snapshots/{commit}/{filename} → local PathBuf
   d. If file exists: read first 8 bytes + header JSON from disk → done (no network)
2. If not cached and --cached is not set: fall back to HTTP Range requests
   a. Build request URL: https://huggingface.co/{repo_id}/resolve/{revision}/{filename}
   b. GET with header `Range: bytes=0-7` → 8-byte response → parse as u64 (header_size)
   c. GET with header `Range: bytes=8-{8+header_size-1}` → JSON header bytes
3. Deserialize JSON into tensor metadata
```

With `--cached`: step 1 only — fail with an error if the file is not in the cache.

The HuggingFace CDN supports Range requests for all files (LFS and git-stored). For gated repos, the `Authorization: Bearer <token>` header is required on both requests.

#### JSON header format

The safetensors JSON header is an object where each key is a tensor name and each value has `dtype`, `shape`, and `data_offsets`:

```json
{
  "__metadata__": { "format": "pt" },
  "model.layers.0.self_attn.q_proj.weight": {
    "dtype": "F8_E4M3",
    "shape": [7168, 7168],
    "data_offsets": [0, 51380224]
  }
}
```

The `__metadata__` key (if present) is extracted separately and stored in `SafetensorsHeaderInfo::metadata`.

#### Cache resolution

The cache-first resolution logic lives in `inspect.rs` (not in `bin/main.rs`), since the library API `inspect_safetensors()` itself benefits from cache-first behavior — any library consumer gets it for free. The separate `inspect_safetensors_local()` function takes a plain `&Path` for callers who already have a file path and don't need cache resolution.

### Library API

**Naming note:** hf-fetch-model uses `SafetensorsHeaderInfo` and `TensorInfo` (not `SafetensorsHeader` / `TensorEntry`) to avoid collision with anamnesis's own types of the same names. anamnesis's `SafetensorsHeader` includes quantization-specific fields (`scheme`, `gptq_config`, `awq_config`) that belong in anamnesis, not here.

**Serialization:** `SafetensorsHeaderInfo`, `TensorInfo`, and `ShardedIndex` derive `serde::Serialize` for `--json` output. `serde` is already a dependency (used by `repo.rs` for API response deserialization).

```rust
/// Metadata for a single tensor from a safetensors header.
///
/// This is hf-fetch-model's own type — lightweight, no quantization logic.
/// Consumers (e.g., anamnesis) map this into their own richer types.
pub struct TensorInfo {
    /// Tensor name (e.g., `"model.layers.0.self_attn.q_proj.weight"`).
    pub name: String,
    /// Element dtype string as it appears in the header (e.g., `"F8_E4M3"`, `"BF16"`).
    pub dtype: String,
    /// Tensor shape (e.g., `[7168, 7168]`).
    pub shape: Vec<usize>,
    /// Byte offset range `[start, end)` within the data section of the file.
    pub data_offsets: (u64, u64),
}

impl TensorInfo {
    /// Total number of elements (product of shape dimensions).
    pub fn num_elements(&self) -> u64;

    /// Byte length of tensor data (`end - start`).
    pub fn byte_len(&self) -> u64;

    /// Bytes per element for the tensor's dtype, if recognized.
    ///
    /// Returns `None` for unknown dtype strings. Recognized dtypes:
    ///
    /// | Dtype string | Bytes | Notes |
    /// |-------------|-------|-------|
    /// | `"BOOL"` | 1 | |
    /// | `"U8"`, `"I8"` | 1 | |
    /// | `"F8_E4M3"`, `"F8_E5M2"` | 1 | FP8 variants |
    /// | `"U16"`, `"I16"`, `"F16"`, `"BF16"` | 2 | |
    /// | `"U32"`, `"I32"`, `"F32"` | 4 | |
    /// | `"U64"`, `"I64"`, `"F64"` | 8 | |
    ///
    /// This saves consumers from reimplementing the same lookup table.
    pub fn dtype_bytes(&self) -> Option<usize>;
}

/// Parsed safetensors header metadata.
pub struct SafetensorsHeaderInfo {
    /// All tensors in the header, in the order they appear in the JSON.
    pub tensors: Vec<TensorInfo>,
    /// Raw `__metadata__` entries, if present.
    ///
    /// For quantized models, this typically contains `quantize_config` entries
    /// (e.g., `bits`, `group_size`, `quant_method`) that consumers like anamnesis
    /// use to distinguish GPTQ from AWQ without downloading weights.
    pub metadata: Option<HashMap<String, String>>,
    /// Size of the JSON header in bytes.
    pub header_size: u64,
    /// Total file size in bytes (header + data), if known.
    ///
    /// **Source:** for local files, from `std::fs::metadata().len()`. For HTTP
    /// Range requests, extracted from the `Content-Range` response header of the
    /// first request (`bytes 0-7/TOTAL` → `TOTAL`). This is free — no extra
    /// request needed.
    pub file_size: Option<u64>,
}

impl SafetensorsHeaderInfo {
    /// Total parameter count across all tensors.
    pub fn total_params(&self) -> u64;

    /// Returns tensors matching a dtype string (e.g., `"F8_E4M3"`).
    pub fn tensors_with_dtype(&self, dtype: &str) -> Vec<&TensorInfo>;
}

/// Inspects a single safetensors file's header (cache-first).
///
/// Checks the local HF cache first. If the file is cached, reads the header
/// from disk with zero network requests. Otherwise, falls back to two HTTP
/// Range requests (8-byte length prefix + JSON header). Does not download
/// tensor data in either case.
///
/// # Errors
///
/// Returns [`FetchError::Http`] if the Range requests fail.
/// Returns [`FetchError::SafetensorsHeader`] if the header is malformed.
pub async fn inspect_safetensors(
    repo_id: &str,
    filename: &str,
    token: Option<&str>,
    revision: Option<&str>,
) -> Result<SafetensorsHeaderInfo, FetchError>;

/// Inspects a single safetensors file's header from a local file path.
///
/// Reads the first `8 + header_size` bytes from disk. Does not read tensor data.
///
/// # Errors
///
/// Returns [`FetchError::Io`] if the file cannot be read.
/// Returns [`FetchError::SafetensorsHeader`] if the header is malformed.
pub fn inspect_safetensors_local(
    path: &Path,
) -> Result<SafetensorsHeaderInfo, FetchError>;

/// Inspects all `*.safetensors` files in a repository (cache-first per file).
///
/// Fetches the file listing via `list_repo_files_with_metadata()`, then
/// inspects each `.safetensors` file's header via [`inspect_safetensors()`].
/// For each file, checks the local cache first and only makes HTTP Range
/// requests on cache miss. Returns full per-shard headers in filename order.
///
/// For a lightweight summary of sharded models (tensor counts per shard
/// without fetching individual headers), use [`fetch_shard_index()`] instead.
/// The CLI uses `fetch_shard_index()` as a fast path when no filename is
/// given and an index file exists.
///
/// # Errors
///
/// Returns [`FetchError::Http`] if the metadata or Range requests fail.
pub async fn inspect_repo_safetensors(
    repo_id: &str,
    token: Option<&str>,
    revision: Option<&str>,
) -> Result<Vec<(String, SafetensorsHeaderInfo)>, FetchError>;
```

### New error variant

```rust
/// Safetensors header is malformed or cannot be parsed.
SafetensorsHeader { filename: String, reason: String },
```

### New module: `inspect.rs`

All safetensors inspection logic lives in a new `src/inspect.rs` module:
- `resolve_cached_path()` — cache-first resolution: `hf_cache_dir()` + `read_ref()` + snapshot directory
- `fetch_header_bytes()` — HTTP Range request helper (two requests); extracts `file_size` from `Content-Range` header
- `parse_header_json()` — JSON deserialization into `Vec<TensorInfo>` + metadata
- `fetch_shard_index()` — fetch and parse `model.safetensors.index.json` (cache-first)
- `inspect_safetensors()`, `inspect_safetensors_local()`, `inspect_repo_safetensors()` — public API

### Sharded model optimization

Many large models are sharded (e.g., DeepSeek-V3 has 163 `.safetensors` files). Naively inspecting every shard means 2×163 = 326 HTTP Range requests. But sharded models include a `model.safetensors.index.json` file that maps every tensor name to its shard filename — all without touching the actual safetensors files. (The index does *not* contain shapes, dtypes, or byte offsets — only the tensor-to-shard assignment.)

The **CLI** uses the shard index as a fast path:

1. No `FILENAME` given: CLI calls `fetch_shard_index()` first. If an index exists, display the summary table (tensor count per shard) from the index alone — **1 request** instead of 326. No Range requests needed.
2. No `FILENAME` given, no index: CLI falls back to `inspect_repo_safetensors()`, which fetches full headers for each shard via concurrent Range requests.
3. `FILENAME` given: CLI calls `inspect_safetensors()` directly — Range requests for the single shard's full header.

The **library** keeps the APIs separate: `fetch_shard_index()` returns a `ShardedIndex` (lightweight, index-only), while `inspect_repo_safetensors()` always returns full `SafetensorsHeaderInfo` per shard (expensive but complete). Callers like anamnesis choose which to call based on their needs.

```rust
/// Parsed `model.safetensors.index.json` for a sharded model.
pub struct ShardedIndex {
    /// Mapping from tensor name to shard filename.
    pub weight_map: HashMap<String, String>,
    /// Ordered list of shard filenames.
    pub shards: Vec<String>,
    /// Raw metadata from the index, if present.
    pub metadata: Option<HashMap<String, serde_json::Value>>,
}

/// Fetches and parses the shard index for a sharded safetensors model (cache-first).
///
/// Returns `Ok(None)` if the repo has no `model.safetensors.index.json` (i.e.,
/// the model is not sharded or uses a single safetensors file).
///
/// # Errors
///
/// Returns [`FetchError::Http`] if the index fetch fails.
/// Returns [`FetchError::SafetensorsHeader`] if the index JSON is malformed.
pub async fn fetch_shard_index(
    repo_id: &str,
    token: Option<&str>,
    revision: Option<&str>,
) -> Result<Option<ShardedIndex>, FetchError>;
```

### Concurrency for multi-file inspection

When the shard index shortcut doesn't apply (no index file, or single-file mode), headers are fetched concurrently with a semaphore (default: 4 concurrent requests). Each header is small (typically 10-100 KiB), so this completes in seconds even for repos with 100+ shards.

---

## Feature 2: `hf-fm du [REPO_ID]`

### Why

`~/.cache/huggingface/hub/` grows unbounded. Users need to know what's taking space before they can decide what to prune (in v0.10.0).

### Behavior

Without arguments: all cached repos, sorted by size descending. With a repo: per-file breakdown. Repos with incomplete downloads show a `PARTIAL` marker.

```
$ hf-fm du
  5.10 GiB  google/gemma-2-2b-it          (8 files)
  2.80 GiB  EleutherAI/pythia-1.4b        (12 files)  PARTIAL
  1.20 GiB  google/gemma-scope-2b-pt-res  (3 files)
  ──────────────────────────────────────────────────
  9.10 GiB  total (3 repos, 23 files)

$ hf-fm du google/gemma-2-2b-it
  2.50 GiB  model-00001-of-00002.safetensors
  2.60 GiB  model-00002-of-00002.safetensors
  1.20 KiB  config.json
  ...
  ──────────────────────────────────────────────────
  5.10 GiB  total (8 files)
```

### Implementation

**Summary mode (no args):** Calls existing `cache::cache_summary()`, sorts by `total_size` descending, formats with `format_size()`. Almost zero new code.

**Per-repo mode:** Walks `<cache_dir>/models--<org>--<name>/snapshots/<commit>/` and lists each file with its size. Reuses `count_snapshot_files()` logic but returns per-file entries instead of totals.

### Library API

```rust
/// Per-file disk usage entry within a cached repository.
pub struct CacheFileUsage {
    /// Filename relative to the snapshot directory.
    pub filename: String,
    /// File size in bytes.
    pub size: u64,
}

/// Returns per-file disk usage for a specific cached repository.
///
/// # Errors
///
/// Returns [`FetchError::Io`] if the cache directory cannot be read.
pub fn cache_repo_usage(repo_id: &str) -> Result<Vec<CacheFileUsage>, FetchError>;
```

The summary-level API already exists as `cache_summary()`.

---

## Non-goals

| Feature | Why not in hf-fetch-model | Where it belongs |
|---------|--------------------------|-----------------|
| Grouped tensor output (e.g., clustering `.qweight`/`.qzeros`/`.scales` triplets) | Requires quantization-specific knowledge (GPTQ vs AWQ grouping rules) | anamnesis CLI (`amn inspect`) — consumes hf-fetch-model's `TensorInfo` and applies its own `TensorRole` classification |
| Quantization scheme detection from tensor patterns | Domain logic, not transport/parsing | anamnesis `parse_safetensors_header()` |
| Dtype enum in hf-fetch-model | Would couple hf-fetch-model to the safetensors spec's dtype list; consumers (anamnesis, candle) have their own enums | `dtype` stays as `String`; `dtype_bytes()` helper covers the common need |

---

## Implementation Order

| Order | Feature | Rationale |
|-------|---------|-----------|
| 1 | `du` | Simplest — reuses `cache_summary()` almost entirely. Ships immediate value while `inspect` is developed. |
| 2 | `inspect` (single file, remote) | Core feature. Implement `fetch_header_bytes()` + `parse_header_json()` + `inspect_safetensors()`. Validates the Range request approach. |
| 3 | `inspect` (cache-first + `--cached`) | Add cache resolution to `inspect_safetensors()`, add `inspect_safetensors_local()`. Reuses the JSON parser from step 2. |
| 4 | `inspect` (all files in repo) | `inspect_repo_safetensors()` with concurrent fetching. Composes the single-file API from step 2 with `list_repo_files_with_metadata()`. |
| 5 | `inspect` (shard index fast path) | Add `fetch_shard_index()` library API + CLI fast path: when no filename is given and an index exists, display summary from index (1 request) instead of fetching all shard headers. |

---

## CLI Tests

Following the v0.8.1 pattern, each feature gets CLI integration tests in `tests/cli.rs`:

- **`du`**: run against `julien-c/dummy-unknown` (cached by existing tests), verify output contains repo name, file count, "total"
- **`du <REPO_ID>`**: verify per-file output contains known filenames
- **`inspect <REPO_ID> <FILE>`**: inspect a small public safetensors model (not `julien-c/dummy-unknown`, which has no `.safetensors` files — use a small model like `hf-internal-testing/tiny-random-BertModel` or similar), verify output contains tensor names, dtypes, shapes, and Size column
- **`inspect <REPO_ID>`**: inspect all safetensors in a repo, verify summary line with file count and total tensors
- **`inspect --cached`**: pre-cache a safetensors file via `download-file`, then inspect from cache, verify output contains `Source: cached`
- **`inspect` metadata display**: inspect a model with `__metadata__`, verify `Metadata:` line appears in output with key=value pairs
- **`inspect --no-metadata`**: inspect the same model, verify `Metadata:` line is absent
- **`inspect --json`**: inspect a safetensors file, verify output is valid JSON, deserialize and check tensor count matches human-readable output
- **`inspect` sharded model**: inspect a small sharded model, verify output mentions `shard index`

---

## Version Milestone

| Version | Content |
|---------|---------|
| v0.9.0 | `inspect` subcommand + `inspect_safetensors()` / `inspect_safetensors_local()` / `inspect_repo_safetensors()` / `fetch_shard_index()` API + `du` subcommand + `cache_repo_usage()` API + `SafetensorsHeaderInfo` / `TensorInfo` / `ShardedIndex` types (all `Serialize`) + `TensorInfo::dtype_bytes()` + `FetchError::SafetensorsHeader` variant + `--json` / `--no-metadata` / `--cached` flags + CLI tests |
| | **Release:** set `version = "0.9.0"` in Cargo.toml, commit, `git tag v0.9.0`, `git push origin v0.9.0` |

---

## Deferred to v0.10.0

The following features from the original v0.9.0 scope are moved to [v0.10.0](v0.10.0-roadmap.md):

| Feature | Rationale for deferral |
|---------|----------------------|
| `verify` (SHA256 re-verification) | Coupled with `clean` — verify-then-prune is a single workflow. Ship together in v0.10.0. |
| `clean` (cache pruning) | Destructive operation needing careful UX (interactive confirmation, `--except`, `--older-than` heuristics). Deserves its own focused release. |
| `du --tree` (cache tree visualization) | Natural companion to `verify` and `clean` — helps users understand cache structure before pruning. |

---

## Appendix A: Dogfooding Feedback

The following enhancements were identified while using `inspect` during the **anamnesis** BnB (BitsAndBytes) quantization work (March 2026). Both are `inspect`-related and are candidates for inclusion in v0.9.x or v0.10.0.

### A.1 `inspect --filter <PATTERN>` — tensor name filtering

**Use case:** "Show me just layer 0 to understand the model structure" — currently requires piping through `grep`, which works but loses the header/summary lines and table formatting.

```
hf-fm inspect Qwen/Qwen3-4B --filter "layers.0"
```

**Implementation:** filter the `Vec<TensorInfo>` by glob or substring match before display. Library API could add an optional `filter` parameter or a post-parse helper. Low effort — the data is already in memory.

### A.2 `inspect --diff <REPO1> <REPO2>` — tensor layout comparison

**Use case:** "Is this NF4 model the same tensor structure as the FP8 variant?" — currently requires inspecting both models separately and manually diffing the output.

```
hf-fm inspect --diff RedHatAI/Llama-3.2-1B-Instruct-FP8 casperhansen/llama-3.2-1b-instruct-awq
```

**Output would show:**
- Tensors only in model A (e.g., `weight_scale_inv` tensors present in FP8 but not AWQ)
- Tensors only in model B (e.g., `qweight`, `qzeros`, `scales` triplets in AWQ but not FP8)
- Tensors in both with different dtypes or shapes
- Tensors in both that match exactly

**Implementation:** inspect both repos, align tensors by name, classify into the four buckets above. Higher effort than `--filter` but unique — no existing tool provides this. Particularly valuable for format research and anamnesis development.

---

## Appendix B: Validation Results

### B.1 Local cache cross-validation (Python)

Tested `inspect --cached` against Python's `safetensors` library on all 39 cached repos:

- **199 files, 16,501 tensors, 0 errors**
- Every tensor name, dtype, shape, `data_offsets`, param count, `header_size`, `file_size`, and metadata entry matched exactly between Python and `hf-fm inspect`
- Script: [`tests/verify_inspect.py`](../../tests/verify_inspect.py)

### B.2 Remote cross-validation (Python)

Tested `inspect` (remote HTTP Range path) against Python `urllib` Range requests on 5 public models not in local cache:

- **3 models with safetensors, 482 tensors, 0 discrepancies**
- 2 models correctly reported "No .safetensors files found" (matching the repo contents)
- Validated: tensor count, names, dtypes, shapes, `data_offsets`, `header_size`, `file_size`, metadata
- Script: [`tests/verify_inspect_remote.py`](../../tests/verify_inspect_remote.py)

### B.3 Python `hf` CLI quirks discovered during testing

| Quirk | Detail |
|-------|--------|
| `--include` silently ignored | `hf download <repo> --include "*.safetensors"` warns and ignores the flag on older versions (v1.3.5) — downloads only non-LFS files |
| Symlink warnings on Windows | Every download emits a long warning about symlinks not being supported, suggesting Developer Mode activation |
| Version nag on every invocation | `hf` prints an upgrade notice on stdout, polluting piped output |

These quirks highlight areas where `hf-fm` already provides a better experience (correct `--filter`/`--preset`, no symlink dependency, clean output).
