# hf-fetch-model v0.10.0 Roadmap — Cache Management

**Date:** March 23, 2026
**Status:** Proposed
**Context:** v0.9.0 added safetensors header inspection (cache-first with HTTP Range fallback, shard index fast path, `__metadata__` display, `--json` for programmatic consumption) and the `du` subcommand (disk usage summary). Users can now *see* what's cached (`list-files --show-cached`, `du`) and *inspect* tensor metadata without downloading full files (`inspect`). v0.10.0 closes the loop: **verify** cached files for integrity, **visualize** cache structure, and **prune** stale entries.

---

## Goal

**Cache management** — three features (`verify`, `clean`, `du --tree`) and corresponding library functions for verifying integrity, visualizing structure, and pruning the local HuggingFace cache.

---

## Existing Infrastructure

v0.10.0 builds on code already in place after v0.9.0:

| Module | What exists | What v0.10.0 adds |
|--------|------------|-------------------|
| `cache.rs` | `hf_cache_dir()`, `cache_summary()` → `Vec<CachedModelSummary>` (repo_id, file_count, total_size, has_partial), `repo_status()` → `RepoStatus` with per-file `FileStatus` (Complete/Partial/Missing), `read_ref()`, `count_snapshot_files()`, `cache_repo_usage()` (v0.9.0) | `verify` reuses `repo_status()` + `checksum.rs`; `clean` adds deletion logic; `du --tree` walks snapshot directories |
| `checksum.rs` | `verify_sha256(path, filename, expected_hex)` — async SHA256 verification | `verify` calls this per file against LFS metadata |
| `repo.rs` | `list_repo_files_with_metadata()` — fetches SHA256 hashes from HF API | `verify` fetches expected hashes via this function |
| `inspect.rs` | `inspect_safetensors()`, `inspect_safetensors_local()`, `inspect_repo_safetensors()`, `fetch_shard_index()`, `resolve_cached_path()` (v0.9.0) | `verify` can reuse `resolve_cached_path()` for local file resolution |
| `bin/main.rs` | `run_status()` / `run_status_all()` — display patterns for per-file and summary tables, `format_size()`, `run_du()`, `run_inspect()` (v0.9.0) | `verify`, `clean`, `du --tree` follow the same display conventions |

---

## Feature 1: `hf-fm verify <REPO_ID> [--revision <REV>] [--token <TOKEN>]`

### Why

Downloaded files can be corrupted (disk errors, interrupted writes, bit rot). After a download, `verify_checksums` runs automatically, but there's no way to re-verify existing cache entries later.

### Behavior

Re-verifies SHA256 checksums of cached files against HF LFS metadata. Requires network (to fetch expected hashes). `REPO_ID` is required — verifying all cached repos at once could take hours for large caches.

```
$ hf-fm verify google/gemma-2-2b-it
  ✓ model-00001-of-00002.safetensors  (2.50 GiB, SHA256 OK)
  ✓ model-00002-of-00002.safetensors  (2.60 GiB, SHA256 OK)
  — config.json                       (1.20 KiB, no LFS hash)
  3 files: 2 SHA256 OK, 1 skipped (no LFS metadata)
```

For corrupted files:

```
$ hf-fm verify google/gemma-2-2b-it
  ✓ model-00001-of-00002.safetensors  (2.50 GiB, SHA256 OK)
  ✗ model-00002-of-00002.safetensors  (2.60 GiB, SHA256 MISMATCH)
  — config.json                       (1.20 KiB, no LFS hash)
  3 files: 1 SHA256 OK, 1 FAILED, 1 skipped
```

### Implementation

1. Fetch expected hashes via `repo::list_repo_files_with_metadata(repo_id, token, revision)`
2. Resolve local paths via `inspect::resolve_cached_path()` (added in v0.9.0) or `cache::hf_cache_dir()` + `read_ref()` + snapshot directory
3. For each file with an LFS SHA256: call `checksum::verify_sha256(path, filename, expected_hex)`
4. Files without LFS metadata (small git-stored files): report "no LFS hash" — skip verification
5. Report summary: verified count, failed count, skipped count

### Library API

```rust
/// Result of verifying a single cached file.
pub struct FileVerification {
    /// Filename within the repository.
    pub filename: String,
    /// File size in bytes.
    pub size: u64,
    /// Verification result.
    pub status: VerifyStatus,
}

/// Verification status for a single file.
#[non_exhaustive]
pub enum VerifyStatus {
    /// SHA256 matches expected hash.
    Ok,
    /// SHA256 does not match (file is corrupted).
    Mismatch { expected: String, actual: String },
    /// File has no LFS metadata (small git-stored file); verification skipped.
    Skipped,
    /// File is missing from cache.
    Missing,
}

/// Verifies SHA256 checksums of cached files against HF LFS metadata.
///
/// # Errors
///
/// Returns [`FetchError::Http`] if the metadata fetch fails.
/// Returns [`FetchError::Io`] if cached files cannot be read.
pub async fn verify_cache(
    repo_id: &str,
    token: Option<&str>,
    revision: Option<&str>,
) -> Result<Vec<FileVerification>, FetchError>;
```

---

## Feature 2: `hf-fm clean <--older-than <DAYS> | --repo <REPO_ID> | --all> [--except <REPO_ID>] [--yes]`

### Why

Once users see disk usage (`du`) and know what's cached, they need a way to reclaim space. Manual deletion of `~/.cache/huggingface/hub/models--*` directories works but is error-prone (partial deletion, leftover refs).

### Behavior

Prune cached repos. **Requires at least one of `--older-than`, `--repo`, or `--all`** — running `clean` with no filter is an error (safety guard against accidental full deletion). Defaults to interactive confirmation. Removes the entire `models--<org>--<name>` directory (snapshots, blobs, refs).

```
$ hf-fm clean --older-than 30
  Will remove:
    EleutherAI/pythia-1.4b  (2.80 GiB, last accessed 45 days ago)
  Keep:
    google/gemma-2-2b-it    (accessed 2 days ago)
  Proceed? [y/N] y
  Removed 1 repo, freed 2.80 GiB

$ hf-fm clean --repo EleutherAI/pythia-1.4b --yes
  Removed EleutherAI/pythia-1.4b (2.80 GiB)

$ hf-fm clean
  error: at least one of --older-than, --repo, or --all is required
```

### Flags

| Flag | Description |
|------|-------------|
| `--older-than <DAYS>` | Only remove repos not accessed within this many days |
| `--repo <REPO_ID>` | Remove a specific repo (repeatable) |
| `--all` | Remove all cached repos (use with `--except` to keep specific ones) |
| `--except <REPO_ID>` | Keep a specific repo even if it matches other filters (repeatable) |
| `--yes` | Skip interactive confirmation |

### Implementation

1. Scan cache via `cache_summary()` to list all repos
2. Apply filters: `--older-than` checks file modification times, `--repo` selects specific repos, `--except` excludes repos
3. Display plan (what will be removed, what kept)
4. If not `--yes`: prompt for confirmation via `stdin`
5. For each repo to remove: `std::fs::remove_dir_all(repo_dir)`
6. Report summary: repos removed, bytes freed

### "Last accessed" heuristic

Use the most recent modification time among all files in the repo's snapshot directory. This is an approximation — HF cache doesn't track access times explicitly — but it's consistent with how `huggingface-cli` determines staleness.

### Library API

```rust
/// Result of a cache clean operation.
pub struct CleanResult {
    /// Repos that were removed.
    pub removed: Vec<CleanedRepo>,
    /// Total bytes freed.
    pub bytes_freed: u64,
}

/// A repo that was removed during cleaning.
pub struct CleanedRepo {
    /// The repository identifier.
    pub repo_id: String,
    /// Bytes freed by removing this repo.
    pub bytes: u64,
}

/// Removes cached repositories matching the given criteria.
///
/// At least one of `repos`, `older_than_days`, or `all` must select
/// repos for removal. `except` overrides any selection.
///
/// # Errors
///
/// Returns [`FetchError::Io`] if directory removal fails.
/// Returns [`FetchError::InvalidArgument`] if no selection criteria provided.
pub fn clean_cache(
    repos: Option<&[String]>,
    older_than_days: Option<u64>,
    all: bool,
    except: &[String],
) -> Result<CleanResult, FetchError>;
```

---

## Feature 3: `hf-fm du --tree`

### Why

Before verifying or pruning, users need to understand the cache's physical structure. The flat `du` output (v0.9.0) shows repo-level and file-level summaries but not the actual directory layout (snapshots, refs, blobs). A tree view makes the HF cache structure legible.

### Behavior

```
$ hf-fm du --tree
  ~/.cache/huggingface/hub/
  ├── models--google--gemma-2-2b-it/                 5.10 GiB
  │   ├── refs/
  │   │   └── main                                   (→ abc1234)
  │   └── snapshots/
  │       └── abc1234def5678.../
  │           ├── model-00001-of-00002.safetensors    2.50 GiB
  │           ├── model-00002-of-00002.safetensors    2.60 GiB
  │           └── config.json                         1.20 KiB
  ├── models--EleutherAI--pythia-1.4b/               2.80 GiB  PARTIAL
  │   └── ...
  └── models--google--gemma-scope-2b-pt-res/         1.20 GiB
      └── ...
  ──────────────────────────────────────────────────
  9.10 GiB total (3 repos, 23 files)
```

With a specific repo:

```
$ hf-fm du --tree google/gemma-2-2b-it
  models--google--gemma-2-2b-it/
  ├── refs/
  │   └── main                                       (→ abc1234)
  └── snapshots/
      └── abc1234def5678.../
          ├── model-00001-of-00002.safetensors        2.50 GiB
          ├── model-00002-of-00002.safetensors        2.60 GiB
          ├── config.json                             1.20 KiB
          ├── tokenizer.json                          4.10 MiB
          └── ...
  ──────────────────────────────────────────────────
  5.10 GiB total (8 files)
```

### Implementation

Walk the `models--<org>--<name>/` directory structure and build a tree representation. Print using box-drawing characters (`├──`, `└──`, `│`). Reuse `format_size()` for file sizes. Show the commit hash that `refs/main` points to (via `read_ref()`).

**No new library API** — `du --tree` is a CLI-only display feature that composes existing functions: `hf_cache_dir()`, `cache_summary()`, `cache_repo_usage()`, `read_ref()`.

---

## Implementation Order

| Order | Feature | Rationale |
|-------|---------|-----------|
| 1 | `verify` | Reuses `checksum.rs` and `repo.rs`. Users need to verify integrity before deciding what to clean. Non-destructive. |
| 2 | `du --tree` | CLI-only display feature, no new library API. Helps users understand cache layout before pruning. |
| 3 | `clean` | Destructive — implement last, after `verify` and `du --tree` are proven. Needs interactive prompt (stdin). |

---

## CLI Tests

Following the v0.8.1 pattern, each feature gets CLI integration tests in `tests/cli.rs`:

- **`verify`**: run against `julien-c/dummy-unknown` (cached by existing tests), verify "SHA256 OK" or "no LFS hash" in output
- **`du --tree`**: run against cached repo, verify output contains box-drawing characters and file sizes
- **`clean --yes`**: test against a repo that can be re-downloaded (e.g., `julien-c/dummy-unknown`), verify "Removed" and "freed" in output
- **`clean` (no filter)**: verify error message about required flags

---

## Version Milestone

| Version | Content |
|---------|---------|
| v0.10.0 | `verify` subcommand + `verify_cache()` API + `clean` subcommand + `clean_cache()` API + `du --tree` flag + `FileVerification` / `VerifyStatus` / `CleanResult` / `CleanedRepo` types + CLI tests |
| | **Release:** set `version = "0.10.0"` in Cargo.toml, commit, `git tag v0.10.0`, `git push origin v0.10.0` |
