SwiLTra-Bench: The Swiss Legal Translation Benchmark
In Switzerland legal translation is uniquely important due to the country's
four official languages and requirements for multilingual legal documentation.
However, this process traditionally relies on professionals who must be both
legal experts and skilled translators -- creating bottlenecks and impacting
effective access to justice. To address this challenge, we introduce
SwiLTra-Bench, a comprehensive multilingual benchmark of over 180K aligned
Swiss legal translation pairs comprising laws, headnotes, and press releases
across all Swiss languages along with English, designed to evaluate LLM-based
translation systems. Our systematic evaluation reveals that frontier models
achieve superior translation performance across all document types, while
specialized translation systems excel specifically in laws but under-perform in
headnotes. Through rigorous testing and human expert validation, we demonstrate
that while fine-tuning open SLMs significantly improves their translation
quality, they still lag behind the best zero-shot prompted frontier models such
as Claude-3.5-Sonnet. Additionally, we present SwiLTra-Judge, a specialized LLM
evaluation system that aligns best with human expert assessments.
67c6bd6e8f3e7fd471affd06
Joel Niklaus
admin_assigned
2025-03-06T13:39:30.778Z
5fae5f68b8423e1d80b8a988
https://cdn-avatars.huggingface.co/v1/production/uploads/1656078368144-5fae5f68b8423e1d80b8a988.jpeg
Joel Niklaus
user
joelniklaus
67c6bd6e8f3e7fd471affd07
Jakob Merane
67c6bd6e8f3e7fd471affd08
Luka Nenadic
admin_assigned
2025-03-06T13:39:40.122Z
66b46ea33a88f464e2842ee8
/avatars/a204b5828a96cf0d5334f238c6c96b13.svg
Luka Nenadic
user
lukanen
67c6bd6e8f3e7fd471affd09
Sina Ahmadi
admin_assigned
2025-03-06T13:39:55.817Z
647530b3e0b188d3cb24b81b
https://cdn-avatars.huggingface.co/v1/production/uploads/647530b3e0b188d3cb24b81b/JzAMifLHLXhjkRSKeF5iQ.jpeg
Sina Ahm
user
SinaAhmadi
67c6bd6e8f3e7fd471affd0a
Yingqiang Gao
admin_assigned
2025-03-06T13:40:05.409Z
645e1d5dad4ded42382ab518
/avatars/a749d27b0d14b80d9fe43eb5bfb58944.svg
Yingqiang Gao
user
CharizardAcademy
67c6bd6e8f3e7fd471affd0b
Cyrill A. H. Chevalley
67c6bd6e8f3e7fd471affd0c
Claude Humbel
67c6bd6e8f3e7fd471affd0d
Christophe Gösken
67c6bd6e8f3e7fd471affd0e
Lorenzo Tanzi
67c6bd6e8f3e7fd471affd0f
Thomas Lüthi
67c6bd6e8f3e7fd471affd10
Stefan Palombo
admin_assigned
2025-03-06T13:52:02.245Z
6594ee0f57a556fbe1720c37
/avatars/662e965fbde86e53b86fd9c2c9dc325a.svg
Stefan Palombo
user
spalombo
67c6bd6e8f3e7fd471affd11
Spencer Poff
admin_assigned
2025-03-06T13:57:18.336Z
63c5cf2176c803a693f59763
https://cdn-avatars.huggingface.co/v1/production/uploads/1678689670011-63c5cf2176c803a693f59763.jpeg
Spencer Poff
user
spencerp
67c6bd6e8f3e7fd471affd12
Boling Yang
admin_assigned
2025-03-06T13:57:11.493Z
65d505dd1e2597ff935d3b03
/avatars/8f7f2bdf0e93a2442b993d064983126f.svg
Boling Yang
user
BolingYang
67c6bd6e8f3e7fd471affd13
Nan Wu
67c6bd6e8f3e7fd471affd14
Matthew Guillod
admin_assigned
2025-03-06T13:57:04.689Z
65d3ae44ea28ba508b5c62fd
/avatars/590cbb68787bd2706a2c2dd711274d61.svg
Matthew Guillod
user
mguillod
67c6bd6e8f3e7fd471affd15
Robin Mamié
admin_assigned
2025-03-06T13:56:57.175Z
64df37df92f3bd97867e2122
https://cdn-avatars.huggingface.co/v1/production/uploads/64df37df92f3bd97867e2122/IfXh4jk5QZ_Pc6tlH8JqY.jpeg
Robin Mamié
user
robinmamie
67c6bd6e8f3e7fd471affd16
Daniel Brunner
admin_assigned
2025-03-06T13:56:50.083Z
6630002ad2f21fa96c0780f2
/avatars/65b33ed0f4afb038fe23ebd98c655a53.svg
Daniel Brunner
user
ravecs
67c6bd6e8f3e7fd471affd17
Julio Pereyra
67c6bd6e8f3e7fd471affd18
Niko Grupen
admin_assigned
2025-03-06T13:56:18.392Z
642c89fb3a6a3fa1ebaf95bc
/avatars/9a1da6fb5e0bf732e1ac29d0722f46db.svg
Niko Grupen
user
nikogrupen
SwiLTra-Bench: The Swiss Legal Translation Benchmark
In Switzerland legal translation is uniquely important due to the country's
four official languages and requirements for multilingual legal documentation.
However, this process traditionally relies on professionals who must be both
legal experts and skilled translators -- creating bottlenecks and impacting
effective access to justice. To address this challenge, we introduce
SwiLTra-Bench, a comprehensive multilingual benchmark of over 180K aligned
Swiss legal translation pairs comprising laws, headnotes, and press releases
across all Swiss languages along with English, designed to evaluate LLM-based
translation systems. Our systematic evaluation reveals that frontier models
achieve superior translation performance across all document types, while
specialized translation systems excel specifically in laws but under-perform in
headnotes. Through rigorous testing and human expert validation, we demonstrate
that while fine-tuning open SLMs significantly improves their translation
quality, they still lag behind the best zero-shot prompted frontier models such
as Claude-3.5-Sonnet. Additionally, we present SwiLTra-Judge, a specialized LLM
evaluation system that aligns best with human expert assessments.
A multilingual benchmark for Swiss legal translation evaluates various models, revealing that while fine-tuned systems improve performance, they do not match zero-shot models like Claude-3.5-Sonnet, particularly in specialized document types.
multilingual benchmark Swiss legal translation pairs LLM-based translation systems frontier models specialized translation systems fine-tuning open SLMs zero-shot prompted frontier models SwiLTra-Judge