The Aloe Family Recipe for Open and Specialized Healthcare LLMs
Purpose: With advancements in Large Language Models (LLMs) for healthcare,
the need arises for competitive open-source models to protect the public
interest. This work contributes to the field of open medical LLMs by optimizing
key stages of data preprocessing and training, while showing how to improve
model safety (through DPO) and efficacy (through RAG). The evaluation
methodology used, which includes four different types of tests, defines a new
standard for the field. The resultant models, shown to be competitive with the
best private alternatives, are released with a permisive license.
  Methods: Building on top of strong base models like Llama 3.1 and Qwen 2.5,
Aloe Beta uses a custom dataset to enhance public data with synthetic Chain of
Thought examples. The models undergo alignment with Direct Preference
Optimization, emphasizing ethical and policy-aligned performance in the
presence of jailbreaking attacks. Evaluation includes close-ended, open-ended,
safety and human assessments, to maximize the reliability of results.
  Results: Recommendations are made across the entire pipeline, backed by the
solid performance of the Aloe Family. These models deliver competitive
performance across healthcare benchmarks and medical fields, and are often
preferred by healthcare professionals. On bias and toxicity, the Aloe Beta
models significantly improve safety, showing resilience to unseen jailbreaking
attacks. For a responsible release, a detailed risk assessment specific to
healthcare is attached to the Aloe Family models.
  Conclusion: The Aloe Beta models, and the recipe that leads to them, are a
significant contribution to the open-source medical LLM field, offering
top-of-the-line performance while maintaining high ethical requirements. This
work sets a new standard for developing and reporting aligned LLMs in
healthcare.
682d83494c4685831f85ec92
Dario Garcia-Gasulla
682d83494c4685831f85ec93
Jordi Bayarri-Planas
extracted_pending
2025-05-21T07:39:54.713Z
661e8559b0338c03bd6e5054
/avatars/044d69afa40ddef4485aebfe984da96b.svg
Bayarri
user
JordiBayarri-bsc
682d83494c4685831f85ec94
Ashwin Kumar Gururajan
682d83494c4685831f85ec95
Enrique Lopez-Cuena
682d83494c4685831f85ec96
Adrian Tormos
extracted_confirmed
2025-05-21T11:04:23.232Z
6622352d9fcf61dee8a1f24d
/avatars/dd164c50b3ecb8861e2294337b942e6f.svg
Adrian Tormos
user
adriantormos
682d83494c4685831f85ec97
Daniel Hinjos
claimed_verified
2025-05-21T08:39:40.380Z
65b8e66f078c543033e49672
/avatars/aa81fd59b4ec624245a4a84d2708962d.svg
Daniel Hinjos García
user
danihinjos
682d83494c4685831f85ec98
Pablo Bernabeu-Perez
extracted_pending
2025-05-21T07:39:54.713Z
620683e7eeb1b73d904c96e5
/avatars/d0309ac9408530a74f1799e175cc5fad.svg
Pablo Bernabeu
user
pabberpe
682d83494c4685831f85ec99
Anna Arias-Duart
extracted_confirmed
2025-05-21T07:59:53.756Z
65d71603ca16ef9ba7fb2efb
/avatars/7d3e1436427f7f58c86fb1f8724c4244.svg
Anna
user
annariasdu
682d83494c4685831f85ec9a
Pablo Agustin Martin-Torres
claimed_verified
2025-05-21T09:03:22.597Z
6565c2a4131d13ccc5df1435
/avatars/218ceac504772e7f0fb3ee4d46d4fab7.svg
Pablo Agustin Martin Torres
user
PabloMartinTorres
682d83494c4685831f85ec9b
Marta Gonzalez-Mallo
682d83494c4685831f85ec9c
Sergio Alvarez-Napagao
extracted_confirmed
2025-05-21T11:38:27.934Z
62d16c742ad06bbc89217797
/avatars/11ce629fbcb33f2431164d8a3e54c876.svg
Sergio Alvarez-Napagao
user
tranchis
682d83494c4685831f85ec9d
Eduard Ayguadé-Parra
682d83494c4685831f85ec9e
Ulises Cortés
The Aloe Family Recipe for Open and Specialized Healthcare LLMs
Purpose: With advancements in Large Language Models (LLMs) for healthcare,
the need arises for competitive open-source models to protect the public
interest. This work contributes to the field of open medical LLMs by optimizing
key stages of data preprocessing and training, while showing how to improve
model safety (through DPO) and efficacy (through RAG). The evaluation
methodology used, which includes four different types of tests, defines a new
standard for the field. The resultant models, shown to be competitive with the
best private alternatives, are released with a permisive license.
  Methods: Building on top of strong base models like Llama 3.1 and Qwen 2.5,
Aloe Beta uses a custom dataset to enhance public data with synthetic Chain of
Thought examples. The models undergo alignment with Direct Preference
Optimization, emphasizing ethical and policy-aligned performance in the
presence of jailbreaking attacks. Evaluation includes close-ended, open-ended,
safety and human assessments, to maximize the reliability of results.
  Results: Recommendations are made across the entire pipeline, backed by the
solid performance of the Aloe Family. These models deliver competitive
performance across healthcare benchmarks and medical fields, and are often
preferred by healthcare professionals. On bias and toxicity, the Aloe Beta
models significantly improve safety, showing resilience to unseen jailbreaking
attacks. For a responsible release, a detailed risk assessment specific to
healthcare is attached to the Aloe Family models.
  Conclusion: The Aloe Beta models, and the recipe that leads to them, are a
significant contribution to the open-source medical LLM field, offering
top-of-the-line performance while maintaining high ethical requirements. This
work sets a new standard for developing and reporting aligned LLMs in
healthcare.
Aloe Beta models improve open-source medical Large Language Models through enhanced preprocessing, Direct Preference Optimization, Retrieval-Augmented Generation, and a rigorous evaluation methodology, achieving competitive performance and high safety standards.
Large Language Models LLMs Direct Preference Optimization DPO Retrieval-Augmented Generation RAG Chain of Thought examples jailbreaking attacks safety assessments human assessments