Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency
  and Performance
In this report, we introduce Falcon-H1, a new series of large language models
(LLMs) featuring hybrid architecture designs optimized for both high
performance and efficiency across diverse use cases. Unlike earlier Falcon
models built solely on Transformer or Mamba architectures, Falcon-H1 adopts a
parallel hybrid approach that combines Transformer-based attention with State
Space Models (SSMs), known for superior long-context memory and computational
efficiency. We systematically revisited model design, data strategy, and
training dynamics, challenging conventional practices in the field. Falcon-H1
is released in multiple configurations, including base and instruction-tuned
variants at 0.5B, 1.5B, 1.5B-deep, 3B, 7B, and 34B parameters. Quantized
instruction-tuned models are also available, totaling over 30 checkpoints on
Hugging Face Hub. Falcon-H1 models demonstrate state-of-the-art performance and
exceptional parameter and training efficiency. The flagship Falcon-H1-34B
matches or outperforms models up to 70B scale, such as Qwen3-32B, Qwen2.5-72B,
and Llama3.3-70B, while using fewer parameters and less data. Smaller models
show similar trends: the Falcon-H1-1.5B-Deep rivals current leading 7B-10B
models, and Falcon-H1-0.5B performs comparably to typical 7B models from 2024.
These models excel across reasoning, mathematics, multilingual tasks,
instruction following, and scientific knowledge. With support for up to 256K
context tokens and 18 languages, Falcon-H1 is suitable for a wide range of
applications. All models are released under a permissive open-source license,
underscoring our commitment to accessible and impactful AI research.
688adbb28b724c8c7187dd70
Jingwei Zuo
claimed_verified
2025-07-31T08:14:55.282Z
6460c3811db65f878513bcaf
https://cdn-avatars.huggingface.co/v1/production/uploads/6460c3811db65f878513bcaf/CRdJ8lXixDku3k8Rm5Stn.jpeg
Jingwei Zuo
user
JingweiZuo
688adbb28b724c8c7187dd71
Maksim Velikanov
688adbb28b724c8c7187dd72
Ilyas Chahed
688adbb28b724c8c7187dd73
Younes Belkada
claimed_verified
2025-07-31T08:14:53.389Z
62441d1d9fdefb55a0b7d12c
https://cdn-avatars.huggingface.co/v1/production/uploads/1648631057413-noauth.png
Younes B
user
ybelkada
688adbb28b724c8c7187dd74
Dhia Eddine Rhayem
688adbb28b724c8c7187dd75
Guillaume Kunsch
688adbb28b724c8c7187dd76
Hakim Hacid
688adbb28b724c8c7187dd77
Hamza Yous
688adbb28b724c8c7187dd78
Brahim Farhat
claimed_verified
2025-07-31T08:14:59.117Z
664c3101d1ba9237d34ae972
/avatars/a830c0ee4c95571419770f1ffb41ef11.svg
BrahimFarhat
user
ifarhat1993
688adbb28b724c8c7187dd79
Ibrahim Khadraoui
688adbb28b724c8c7187dd7a
Mugariya Farooq
688adbb28b724c8c7187dd7b
Giulia Campesan
688adbb28b724c8c7187dd7c
Ruxandra Cojocaru
688adbb28b724c8c7187dd7d
Yasser Djilali
688adbb28b724c8c7187dd7e
Shi Hu
688adbb28b724c8c7187dd7f
Iheb Chaabane
claimed_verified
2025-07-31T08:14:57.046Z
660bd11884eca4537c4aeedd
/avatars/10fab6ec552f3d34ceac2bf01df32975.svg
Iheb Chaabane
user
Iheb-Chaabane
688adbb28b724c8c7187dd80
Puneesh Khanna
688adbb28b724c8c7187dd81
Mohamed El Amine Seddik
claimed_verified
2025-07-31T21:00:56.740Z
6460bb7c455531c6be7997bd
https://cdn-avatars.huggingface.co/v1/production/uploads/6460bb7c455531c6be7997bd/l0uuYf0rAPnKfvS90jAAr.jpeg
Mohamed El Amine Seddik
user
melaseddik
688adbb28b724c8c7187dd82
Ngoc Dung Huynh
688adbb28b724c8c7187dd83
Phuc Le Khac
688adbb28b724c8c7187dd84
Leen AlQadi
688adbb28b724c8c7187dd85
Billel Mokeddem
688adbb28b724c8c7187dd86
Mohamed Chami
688adbb28b724c8c7187dd87
Abdalgader Abubaker
688adbb28b724c8c7187dd88
Mikhail Lubinets
688adbb28b724c8c7187dd89
Kacper Piskorski
claimed_verified
2025-07-31T21:00:58.823Z
65d4bd5829b4ac81c2637518
https://cdn-avatars.huggingface.co/v1/production/uploads/65d4bd5829b4ac81c2637518/iBEQ7jCz5Z-a1KNn_03h6.png
Kasper Piskorski
user
kasper-piskorski
688adbb28b724c8c7187dd8a
Slim Frikha
claimed_verified
2025-07-31T21:01:01.709Z
66db041dd04c920ebf3198ff
https://cdn-avatars.huggingface.co/v1/production/uploads/66db041dd04c920ebf3198ff/yfkeoBpp6ztKi27GglP1z.jpeg
Slim Frikha
user
slimfrikha
Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency
  and Performance
In this report, we introduce Falcon-H1, a new series of large language models
(LLMs) featuring hybrid architecture designs optimized for both high
performance and efficiency across diverse use cases. Unlike earlier Falcon
models built solely on Transformer or Mamba architectures, Falcon-H1 adopts a
parallel hybrid approach that combines Transformer-based attention with State
Space Models (SSMs), known for superior long-context memory and computational
efficiency. We systematically revisited model design, data strategy, and
training dynamics, challenging conventional practices in the field. Falcon-H1
is released in multiple configurations, including base and instruction-tuned
variants at 0.5B, 1.5B, 1.5B-deep, 3B, 7B, and 34B parameters. Quantized
instruction-tuned models are also available, totaling over 30 checkpoints on
Hugging Face Hub. Falcon-H1 models demonstrate state-of-the-art performance and
exceptional parameter and training efficiency. The flagship Falcon-H1-34B
matches or outperforms models up to 70B scale, such as Qwen3-32B, Qwen2.5-72B,
and Llama3.3-70B, while using fewer parameters and less data. Smaller models
show similar trends: the Falcon-H1-1.5B-Deep rivals current leading 7B-10B
models, and Falcon-H1-0.5B performs comparably to typical 7B models from 2024.
These models excel across reasoning, mathematics, multilingual tasks,
instruction following, and scientific knowledge. With support for up to 256K
context tokens and 18 languages, Falcon-H1 is suitable for a wide range of
applications. All models are released under a permissive open-source license,
underscoring our commitment to accessible and impactful AI research.
Falcon-H1, a new series of large language models with a hybrid architecture combining Transformer-based attention and State Space Models, achieves state-of-the-art performance and efficiency across various tasks and sizes.
large language models hybrid architecture Transformer-based attention State Space Models long-context memory computational efficiency model design data strategy training dynamics instruction-tuned quantized models parameter efficiency training efficiency context tokens multilingual tasks instruction following scientific knowledge open-source license