# Benchmark History

Auto-generated by the [Benchmark workflow](.github/workflows/benchmark.yml).

## WER / Quality Regression Tracking

| Dataset | Samples | WER | Notes |
|---|---|---|---|
| **GigaSpeech2-vi (clean)** | — | **~7.7%** | Published upstream benchmark on clean Vietnamese speech |
| **FLEURS Vietnamese** | 857 | **103.08%** | Foreign names, numbers, and English terms transcribed phonetically into Vietnamese |

> The FLEURS baseline is **intentionally high** — the dataset contains many proper names, digits, and English loanwords that the model transcribes phonetically into Vietnamese. It is used for **regression tracking only** (threshold: 1.05), not as an absolute quality metric. For production-quality assessment, refer to the GigaSpeech2-vi benchmark above.

## 2026-05-05 (local run, commit latest)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 59.92 ms | 62.42× | — |
| CoreML | 93.15 ms | 40.15× | — |

## 2026-05-05 (commit ff77a30)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF |
|---|---|---|
| CPU | 61.21 ms | 61.11× |
| CoreML | 91.20 ms | 41.01× |

## 2026-05-05 (commit 530115a)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF |
|---|---|---|
| CPU | 84.55 ms | 44.24 |
| CoreML | 101.28 ms | 36.93 |

## 2026-05-05 (commit 002ae9a)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 83.86 ms | 44.60 | 1620.7 MB |
| CoreML | 126.60 ms | 29.54 | 1649.1 MB |

## 2026-05-05 (commit 116beed)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 85.62 ms | 43.68 | 1664.1 MB |
| CoreML |  ms |  | 1650.5 MB |

## 2026-05-05 (commit 0ea2b1a)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 103.73 ms | 36.06 | 1674.5 MB |
| CoreML | 117.80 ms | 31.75 | 1669.7 MB |

## 2026-05-05 (commit bd5145c)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 80.19 ms | 46.64 | 1646.6 MB |
| CoreML | 106.86 ms | 35.00 | 1676.0 MB |

## 2026-05-06 (commit 49b6d26)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 76.76 ms | 48.73 | 1675.2 MB |
| CoreML | 98.60 ms | 37.93 | 1676.2 MB |

## 2026-05-06 (commit 1e89f83)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 70.76 ms | 52.85 | 1658.7 MB |
| CoreML | 98.71 ms | 37.89 | 1675.7 MB |

## 2026-05-06 (commit 13a94f8)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 82.47 ms | 45.35 | 1675.7 MB |
| CoreML | 109.58 ms | 34.13 | 1666.4 MB |

## 2026-05-06 (commit 537c98b)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 75.99 ms | 49.22 | 1636.6 MB |
| CoreML | 113.01 ms | 33.10 | 1675.8 MB |

## 2026-05-06 (commit 47a44aa)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 72.83 ms | 51.35 | 1639.1 MB |
| CoreML | 104.17 ms | 35.90 | 1651.5 MB |

## 2026-05-06 (commit bc6d08a)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 75.57 ms | 49.49 | 1636.1 MB |
| CoreML | 102.28 ms | 36.57 | 1667.4 MB |

## 2026-05-06 (commit 3c07f75)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 75.49 ms | 49.54 | 1652.3 MB |
| CoreML | 101.18 ms | 36.96 | 1666.4 MB |

## 2026-05-06 (commit 29c7146)

### macOS-14 (Apple Silicon)

| Backend | Mean Latency | RTF | Peak RSS |
|---|---|---|---|
| CPU | 77.99 ms | 47.95 | 1664.5 MB |
| CoreML | 92.64 ms | 40.37 | 1665.6 MB |
