Thanks a lot for the hints! I’m also using 10% splits of the dataset for both training and testing, and evaluating on the full test set afterwards. Still, I’m having problems evaluating WER on the test dataset after a training run. I wrote a small helper function to compute wer from chunks, in CPU:
import jiwer

def chunked_wer(targets, predictions, chunk_size=None):
    if chunk_size is None: return jiwer.wer(targets, predictions)
    start = 0
    end = chunk_size
    H, S, D, I = 0, 0, 0, 0
    while start < len(targets):
        chunk_metrics = jiwer.compute_measures(targets[start:end], predictions[start:end])
        H = H + chunk_metrics["hits"]
        S = S + chunk_metrics["substitutions"]
        D = D + chunk_metrics["deletions"]
        I = I + chunk_metrics["insertions"]
        start += chunk_size
        end += chunk_size
    return float(S + D + I) / float(H + S + D)

# Your targets and predictions
target = result["target_text"]
preds = result["pred_strings"]

print("Total (chunk=5000), WER: {:2f}".format(100 * chunked_wer(target, preds, chunk_size=5000)))