Aurora-M: The First Open Source Multilingual Language Model Red-teamed
  according to the U.S. Executive Order
Pretrained language models underpin several AI applications, but their high
computational cost for training limits accessibility. Initiatives such as BLOOM
and StarCoder aim to democratize access to pretrained models for collaborative
community development. However, such existing models face challenges: limited
multilingual capabilities, continual pretraining causing catastrophic
forgetting, whereas pretraining from scratch is computationally expensive, and
compliance with AI safety and development laws. This paper presents Aurora-M, a
15B parameter multilingual open-source model trained on English, Finnish,
Hindi, Japanese, Vietnamese, and code. Continually pretrained from
StarCoderPlus on 435 billion additional tokens, Aurora-M surpasses 2 trillion
tokens in total training token count. It is the first open-source multilingual
model fine-tuned on human-reviewed safety instructions, thus aligning its
development not only with conventional red-teaming considerations, but also
with the specific concerns articulated in the Biden-Harris Executive Order on
the Safe, Secure, and Trustworthy Development and Use of Artificial
Intelligence. Aurora-M is rigorously evaluated across various tasks and
languages, demonstrating robustness against catastrophic forgetting and
outperforming alternatives in multilingual settings, particularly in safety
evaluations. To promote responsible open-source LLM development, Aurora-M and
its variants are released at
https://huggingface.co/collections/aurora-m/aurora-m-models-65fdfdff62471e09812f5407 .
660b5b0becc38ac2450f232c
Taishi Nakamura
claimed_verified
2024-04-05T10:05:04.200Z
6308c49c454dc257521bc7f9
https://cdn-avatars.huggingface.co/v1/production/uploads/6308c49c454dc257521bc7f9/UWUS6OPa6OpVu1T0gd-wJ.jpeg
Taishi
user
Taishi-N324
660b5b0becc38ac2450f232d
Mayank Mishra
claimed_verified
2024-04-02T11:05:31.391Z
62cd5057674cdb524450093d
https://cdn-avatars.huggingface.co/v1/production/uploads/62cd5057674cdb524450093d/f67rlrdsKPRTLdXCXoa_X.jpeg
Mayank Mishra
user
mayank-mishra
660b5b0becc38ac2450f232e
Simone Tedeschi
claimed_verified
2024-04-02T11:07:48.747Z
61b85aa99ba538c73a7dc78b
https://cdn-avatars.huggingface.co/v1/production/uploads/61b85aa99ba538c73a7dc78b/gWxtQAvOYn7cXgE_nAy0p.jpeg
Simone Tedeschi
user
sted97
660b5b0becc38ac2450f232f
Yekun Chai
admin_assigned
2024-04-02T15:11:18.357Z
626a6ce8adf559c88de9ace0
https://cdn-avatars.huggingface.co/v1/production/uploads/1651142275247-626a6ce8adf559c88de9ace0.jpeg
CYK
user
cyk1337
660b5b0becc38ac2450f2330
Jason T Stillerman
claimed_verified
2024-04-03T09:26:11.333Z
6178d842267320abb99df776
/avatars/8739705ef7eabc945a1ef3c07196af13.svg
Jason Stillerman
user
stillerman
660b5b0becc38ac2450f2331
Felix Friedrich
claimed_verified
2025-06-20T12:21:21.925Z
62e7dd4036a8e8a82700041c
https://cdn-avatars.huggingface.co/v1/production/uploads/62e7dd4036a8e8a82700041c/Dgk9mXYLVd4LpiNLWjn-q.jpeg
Felix Friedrich
user
felfri
660b5b0becc38ac2450f2332
Prateek Yadav
660b5b0becc38ac2450f2333
Tanmay Laud
claimed_verified
2024-04-03T09:26:26.444Z
5fd85dbada7d9193d8b05b69
https://cdn-avatars.huggingface.co/v1/production/uploads/1624471791171-5fd85dbada7d9193d8b05b69.png
Tanmay Laud
user
tanmaylaud
660b5b0becc38ac2450f2334
Vu Minh Chien
claimed_verified
2024-04-02T19:35:16.724Z
60535c9d10aba34e3b6a2ef7
https://cdn-avatars.huggingface.co/v1/production/uploads/1616075850230-noauth.jpeg
vumichien
user
vumichien
660b5b0becc38ac2450f2335
Terry Yue Zhuo
660b5b0becc38ac2450f2336
Diganta Misra
extracted_confirmed
2024-04-02T01:15:26.036Z
643006f01572f43a481766a9
https://cdn-avatars.huggingface.co/v1/production/uploads/643006f01572f43a481766a9/egLlzmsWxHovmuvqAPnBO.jpeg
_
user
Xa9aX
660b5b0becc38ac2450f2337
Ben Bogin
660b5b0becc38ac2450f2338
Xuan-Son Vu
660b5b0becc38ac2450f2339
Marzena Karpinska
claimed_verified
2024-07-01T07:47:18.567Z
62293b03acd5bef90e55c4ae
https://cdn-avatars.huggingface.co/v1/production/uploads/1667593978543-62293b03acd5bef90e55c4ae.png
Marzena Karpinska
user
marzena
660b5b0becc38ac2450f233a
Arnav Varma Dantuluri
claimed_verified
2024-04-08T08:12:24.997Z
6303e701d14428368d19569e
/avatars/8fb08261884c5fc61925a25594161f13.svg
Arnav Dantuluri
user
Aldrich
660b5b0becc38ac2450f233b
Wojciech Kusa
claimed_verified
2025-06-11T09:23:29.694Z
611669ad267478fd54891f8d
https://cdn-avatars.huggingface.co/v1/production/uploads/1659021160775-611669ad267478fd54891f8d.jpeg
Wojciech Kusa
user
WojciechKusa
660b5b0becc38ac2450f233c
Tommaso Furlanello
660b5b0becc38ac2450f233d
Rio Yokota
660b5b0becc38ac2450f233e
Niklas Muennighoff
660b5b0becc38ac2450f233f
Suhas Pai
660b5b0becc38ac2450f2340
Tosin Adewumi
660b5b0becc38ac2450f2341
Veronika Laippala
660b5b0becc38ac2450f2342
Xiaozhe Yao
claimed_verified
2024-04-03T09:25:59.978Z
61919e59d221f4281b3833d5
https://cdn-avatars.huggingface.co/v1/production/uploads/1669164541734-61919e59d221f4281b3833d5.jpeg
Xiaozhe Yao
user
xzyao
660b5b0becc38ac2450f2343
Adalberto Junior
660b5b0becc38ac2450f2344
Alpay Ariyak
claimed_verified
2024-04-03T09:25:39.066Z
63972847b3e2256c9ce1307b
https://cdn-avatars.huggingface.co/v1/production/uploads/63972847b3e2256c9ce1307b/Zt-O-F_gl0dDefbkOORfY.png
Alpay Ariyak
user
alpayariyak
660b5b0becc38ac2450f2345
Aleksandr Drozd
660b5b0becc38ac2450f2346
Jordan Clive
claimed_verified
2024-04-02T11:05:14.866Z
608995d619137b3a6ba76113
https://cdn-avatars.huggingface.co/v1/production/uploads/608995d619137b3a6ba76113/QK25kNi5XIlw8m3104KC8.jpeg
Jordan Clive
user
jordiclive
660b5b0becc38ac2450f2347
Kshitij Gupta
claimed_verified
2024-04-02T11:03:01.538Z
625daa0c943e346492b63d21
/avatars/860f8e9bff85325096b6f8e15102def6.svg
Kshitij Gupta
user
kshitijkg
660b5b0becc38ac2450f2348
Liangyu Chen
660b5b0becc38ac2450f2349
Qi Sun
claimed_verified
2024-08-05T09:02:30.547Z
644b983f0fbe4830f192c4f5
https://cdn-avatars.huggingface.co/v1/production/uploads/644b983f0fbe4830f192c4f5/4eX4OS0gnXGqLu3eaMml_.png
Qi Sun
user
lfsm
660b5b0becc38ac2450f234a
Ken Tsui
claimed_verified
2024-04-02T19:35:06.873Z
60e50ce5350d181892d5a636
https://cdn-avatars.huggingface.co/v1/production/uploads/60e50ce5350d181892d5a636/qjYikZtE-2kjub-6XG5Kn.jpeg
Ken Tsui
user
kenhktsui
660b5b0becc38ac2450f234b
Noah Persaud
660b5b0becc38ac2450f234c
Nour Fahmy
660b5b0becc38ac2450f234d
Tianlong Chen
660b5b0becc38ac2450f234e
Mohit Bansal
660b5b0becc38ac2450f234f
Nicolo Monti
claimed_verified
2024-04-02T19:35:13.690Z
62f3f80d4794a3c919bca0d5
https://cdn-avatars.huggingface.co/v1/production/uploads/62f3f80d4794a3c919bca0d5/R-OuVhyE5KwvuHx9hUhfu.png
Nicolò Monti
user
nicolomonti
660b5b0becc38ac2450f2350
Tai Dang
660b5b0becc38ac2450f2351
Ziyang Luo
claimed_verified
2024-04-02T19:35:09.791Z
6090ff099a8bcaa437b234a4
https://cdn-avatars.huggingface.co/v1/production/uploads/6090ff099a8bcaa437b234a4/iUvw7JXT-ngI7rGk1x-io.jpeg
Ziyang Luo
user
Ziyang
660b5b0becc38ac2450f2352
Tien-Tung Bui
660b5b0becc38ac2450f2353
Roberto Navigli
660b5b0becc38ac2450f2354
Virendra Mehta
claimed_verified
2024-09-23T13:07:20.289Z
604eb19e3050a33ebb17ef58
/avatars/23650002ba3befee83060fe978a251c8.svg
Virendra Mehta
user
Veeru
660b5b0becc38ac2450f2355
Matthew Blumberg
660b5b0becc38ac2450f2356
Victor May
claimed_verified
2024-04-03T09:25:24.578Z
627bda4f1b02d7b5bf63f1f8
https://cdn-avatars.huggingface.co/v1/production/uploads/627bda4f1b02d7b5bf63f1f8/sqK5uRemxbv-zBQ9B8ugE.jpeg
VM
user
cabbage972
660b5b0becc38ac2450f2357
Huu Nguyen
extracted_confirmed
2024-04-02T02:11:32.098Z
5fc6879e1c5ee87b1164876d
https://cdn-avatars.huggingface.co/v1/production/uploads/5fc6879e1c5ee87b1164876d/Tjnm_lv0Bq0gPbFOTDH6E.jpeg
Huu Nguyen
user
huu-ontocord
660b5b0becc38ac2450f2358
Sampo Pyysalo
extracted_confirmed
2024-04-02T08:08:30.308Z
602fb2d93d343696d96ef287
/avatars/a78ce59506a416e3df1d283a13e5aefa.svg
Sampo Pyysalo
user
spyysalo
Aurora-M: The First Open Source Multilingual Language Model Red-teamed
  according to the U.S. Executive Order
Pretrained language models underpin several AI applications, but their high
computational cost for training limits accessibility. Initiatives such as BLOOM
and StarCoder aim to democratize access to pretrained models for collaborative
community development. However, such existing models face challenges: limited
multilingual capabilities, continual pretraining causing catastrophic
forgetting, whereas pretraining from scratch is computationally expensive, and
compliance with AI safety and development laws. This paper presents Aurora-M, a
15B parameter multilingual open-source model trained on English, Finnish,
Hindi, Japanese, Vietnamese, and code. Continually pretrained from
StarCoderPlus on 435 billion additional tokens, Aurora-M surpasses 2 trillion
tokens in total training token count. It is the first open-source multilingual
model fine-tuned on human-reviewed safety instructions, thus aligning its
development not only with conventional red-teaming considerations, but also
with the specific concerns articulated in the Biden-Harris Executive Order on
the Safe, Secure, and Trustworthy Development and Use of Artificial
Intelligence. Aurora-M is rigorously evaluated across various tasks and
languages, demonstrating robustness against catastrophic forgetting and
outperforming alternatives in multilingual settings, particularly in safety
evaluations. To promote responsible open-source LLM development, Aurora-M and
its variants are released at
https://huggingface.co/collections/aurora-m/aurora-m-models-65fdfdff62471e09812f5407 .
Aurora-M, a 15B parameter multilingual open-source model trained on multiple languages and code, addresses challenges of catastrophic forgetting and AI safety through continual pretraining and human-reviewed safety instructions.
pretrained language models multilingual capabilities continual pretraining catastrophic forgetting open-source model StarCoderPlus human-reviewed safety instructions Biden-Harris Executive Order robustness safety evaluations