translated sentences are to the target sentences in Flores 200. We sample at temperature T = 1 and score
using SacreBLEU v2.3.0 [19] with the Flores 200 tokenizer.
We view this evaluation as a rough indicator of which languages our model is probably better and worse at —
small differences between similar scores could be due to noise. Similarly, [20] suggests bucketing the scores
as follows: “[s]cores over 30 generally reflect understandable translations” and “[s]cores over 50 generally
reflect good and fluent translations.”
ClaudeModels