8

Table 6: Effective robustness across various datasets. WER results for one ID reference dataset
and four OOD datasets. The relative error rate (RER) is shown on the right, giving the per-dataset
effective robustness scores. The macro-average results are shown in the bottom row.

Dataset
GigaSpeech
CHIME-4
Earnings-22
FLEURS
SPGISpeech
Average

large-v2
10.7
11.8
16.6
4.2
3.8
9.4

distil-large-v2 RER
-2.0
18.4
1.6
48.2
-12.8
10.7

10.5
14.0
16.9
6.3
3.3
10.2

Table 7: Comparison of long-form transcription algorithms. Average WER results over the four
OOD long-form test sets for the sequential and chunked long-form algorithms. Relative latency is
the inference time relative to the large-v2 model with sequential long-form decoding. The chunked
transcription results are reported using a batch size of 16.

Algorithm Rel. Latency Avg. OOD WER
Model
10.4
Sequential
large-v2
Chunked
11.7
large-v2
11.6
distil-large-v2 Chunked

1.0
9.9
57.5

8.2 LONG-FORM EVALUATION
DISTIL-WHISPER