Llama-Nemotron: Efficient Reasoning Models
We introduce the Llama-Nemotron series of models, an open family of
heterogeneous reasoning models that deliver exceptional reasoning capabilities,
inference efficiency, and an open license for enterprise use. The family comes
in three sizes -- Nano (8B), Super (49B), and Ultra (253B) -- and performs
competitively with state-of-the-art reasoning models such as DeepSeek-R1 while
offering superior inference throughput and memory efficiency. In this report,
we discuss the training procedure for these models, which entails using neural
architecture search from Llama 3 models for accelerated inference, knowledge
distillation, and continued pretraining, followed by a reasoning-focused
post-training stage consisting of two main parts: supervised fine-tuning and
large scale reinforcement learning. Llama-Nemotron models are the first
open-source models to support a dynamic reasoning toggle, allowing users to
switch between standard chat and reasoning modes during inference. To further
support open research and facilitate model development, we provide the
following resources: 1. We release the Llama-Nemotron reasoning models --
LN-Nano, LN-Super, and LN-Ultra -- under the commercially permissive NVIDIA
Open Model License Agreement. 2. We release the complete post-training dataset:
Llama-Nemotron-Post-Training-Dataset. 3. We also release our training
codebases: NeMo, NeMo-Aligner, and Megatron-LM.
681885e585df02e13b44d3f1
Akhiad Bercovich
681885e585df02e13b44d3f2
Itay Levy
681885e585df02e13b44d3f3
Izik Golan
681885e585df02e13b44d3f4
Mohammad Dabbah
681885e585df02e13b44d3f5
Ran El-Yaniv
681885e585df02e13b44d3f6
Omri Puny
681885e585df02e13b44d3f7
Ido Galil
681885e585df02e13b44d3f8
Zach Moshe
681885e585df02e13b44d3f9
Tomer Ronen
681885e585df02e13b44d3fa
Najeeb Nabwani
681885e585df02e13b44d3fb
Ido Shahaf
681885e585df02e13b44d3fc
Oren Tropp
681885e585df02e13b44d3fd
Ehud Karpas
681885e585df02e13b44d3fe
Ran Zilberstein
681885e585df02e13b44d3ff
Jiaqi Zeng
681885e585df02e13b44d400
Soumye Singhal
claimed_verified
2025-05-06T08:34:23.674Z
623a5318d2f0273325e008b9
https://cdn-avatars.huggingface.co/v1/production/uploads/1668579465278-623a5318d2f0273325e008b9.jpeg
Soumye Singhal
user
soumye
681885e585df02e13b44d401
Alexander Bukharin
681885e585df02e13b44d402
Yian Zhang
681885e585df02e13b44d403
Tugrul Konuk
681885e585df02e13b44d404
Gerald Shen
681885e585df02e13b44d405
Ameya Sunil Mahabaleshwarkar
681885e585df02e13b44d406
Bilal Kartal
681885e585df02e13b44d407
Yoshi Suhara
681885e585df02e13b44d408
Olivier Delalleau
681885e585df02e13b44d409
Zijia Chen
681885e585df02e13b44d40a
Zhilin Wang
681885e585df02e13b44d40b
David Mosallanezhad
681885e585df02e13b44d40c
Adi Renduchintala
681885e585df02e13b44d40d
Haifeng Qian
681885e585df02e13b44d40e
Dima Rekesh
681885e585df02e13b44d40f
Fei Jia
681885e585df02e13b44d410
Somshubra Majumdar
681885e585df02e13b44d411
Vahid Noroozi
681885e585df02e13b44d412
Wasi Uddin Ahmad
681885e585df02e13b44d413
Sean Narenthiran
681885e585df02e13b44d414
Aleksander Ficek
681885e585df02e13b44d415
Mehrzad Samadi
681885e585df02e13b44d416
Jocelyn Huang
681885e585df02e13b44d417
Siddhartha Jain
681885e585df02e13b44d418
Igor Gitman
681885e585df02e13b44d419
Ivan Moshkov
claimed_verified
2025-05-06T08:34:29.021Z
65cd2166c52afc14cebc171a
https://cdn-avatars.huggingface.co/v1/production/uploads/65cd2166c52afc14cebc171a/LhIg07Yaqo0gmqYzGCcTe.png
Ivan Moshkov
user
imoshkov
681885e585df02e13b44d41a
Wei Du
claimed_verified
2025-05-07T08:09:46.305Z
662fcd0314e87951eaded842
/avatars/868eccf33126298bd47ecac9da810116.svg
Wei Du
user
woshiweidu
681885e585df02e13b44d41b
Shubham Toshniwal
claimed_verified
2025-05-06T08:34:26.150Z
65c55fbe488606617e9dbabd
https://cdn-avatars.huggingface.co/v1/production/uploads/65c55fbe488606617e9dbabd/42xMtIIPZPCoF-bYHYMDz.jpeg
Shubham Toshniwal
user
stoshniwal
681885e585df02e13b44d41c
George Armstrong
681885e585df02e13b44d41d
Branislav Kisacanin
681885e585df02e13b44d41e
Matvei Novikov
681885e585df02e13b44d41f
Daria Gitman
681885e585df02e13b44d420
Evelina Bakhturina
claimed_verified
2025-06-11T16:02:41.162Z
64ecf888961565b46783bf08
/avatars/0c4ac90b7fa8a6091a67b0216dada7db.svg
Evelina Bakhturina
user
ekmb
681885e585df02e13b44d421
Jane Polak Scowcroft
681885e585df02e13b44d422
John Kamalu
681885e585df02e13b44d423
Dan Su
claimed_verified
2025-06-05T07:29:15.106Z
67e34615c15564fd97f697ce
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/BNXjsEA5rGn7uFoFuVkB6.png
Dan Su
user
sudandandansu1
681885e585df02e13b44d424
Kezhi Kong
681885e585df02e13b44d425
Markus Kliegl
681885e585df02e13b44d426
Rabeeh Karimi
681885e585df02e13b44d427
Ying Lin
681885e585df02e13b44d428
Sanjeev Satheesh
681885e585df02e13b44d429
Jupinder Parmar
681885e585df02e13b44d42a
Pritam Gundecha
681885e585df02e13b44d42b
Brandon Norick
681885e585df02e13b44d42c
Joseph Jennings
681885e585df02e13b44d42d
Shrimai Prabhumoye
681885e585df02e13b44d42e
Syeda Nahida Akter
681885e585df02e13b44d42f
Mostofa Patwary
681885e585df02e13b44d430
Abhinav Khattar
681885e585df02e13b44d431
Deepak Narayanan
681885e585df02e13b44d432
Roger Waleffe
681885e585df02e13b44d433
Jimmy Zhang
681885e585df02e13b44d434
Bor-Yiing Su
681885e585df02e13b44d435
Guyue Huang
681885e585df02e13b44d436
Terry Kong
681885e585df02e13b44d437
Parth Chadha
681885e585df02e13b44d438
Sahil Jain
681885e585df02e13b44d439
Christine Harvey
681885e585df02e13b44d43a
Elad Segal
claimed_verified
2025-05-15T10:33:15.041Z
5f5b0efe10b2753d9000c888
https://cdn-avatars.huggingface.co/v1/production/uploads/1628140531144-5f5b0efe10b2753d9000c888.jpeg
Elad Segal
user
eladsegal
681885e585df02e13b44d43b
Jining Huang
681885e585df02e13b44d43c
Sergey Kashirsky
681885e585df02e13b44d43d
Robert McQueen
681885e585df02e13b44d43e
Izzy Putterman
681885e585df02e13b44d43f
George Lam
681885e585df02e13b44d440
Arun Venkatesan
681885e585df02e13b44d441
Sherry Wu
681885e585df02e13b44d442
Vinh Nguyen
681885e585df02e13b44d443
Manoj Kilaru
681885e585df02e13b44d444
Andrew Wang
681885e585df02e13b44d445
Anna Warno
681885e585df02e13b44d446
Abhilash Somasamudramath
681885e585df02e13b44d447
Sandip Bhaskar
681885e585df02e13b44d448
Maka Dong
681885e585df02e13b44d449
Nave Assaf
681885e585df02e13b44d44a
Shahar Mor
681885e585df02e13b44d44b
Omer Ullman Argov
681885e585df02e13b44d44c
Scot Junkin
681885e585df02e13b44d44d
Oleksandr Romanenko
681885e585df02e13b44d44e
Pedro Larroy
681885e585df02e13b44d44f
Monika Katariya
681885e585df02e13b44d450
Marco Rovinelli
681885e585df02e13b44d451
Viji Balas
681885e585df02e13b44d452
Nicholas Edelman
681885e585df02e13b44d453
Anahita Bhiwandiwalla
681885e585df02e13b44d454
Muthu Subramaniam
681885e585df02e13b44d455
Smita Ithape
681885e585df02e13b44d456
Karthik Ramamoorthy
681885e585df02e13b44d457
Yuting Wu
681885e585df02e13b44d458
Suguna Varshini Velury
681885e585df02e13b44d459
Omri Almog
681885e585df02e13b44d45a
Joyjit Daw
681885e585df02e13b44d45b
Denys Fridman
681885e585df02e13b44d45c
Erick Galinkin
681885e585df02e13b44d45d
Michael Evans
681885e585df02e13b44d45e
Katherine Luna
681885e585df02e13b44d45f
Leon Derczynski
681885e585df02e13b44d460
Nikki Pope
681885e585df02e13b44d461
Eileen Long
681885e585df02e13b44d462
Seth Schneider
681885e585df02e13b44d463
Guillermo Siman
681885e585df02e13b44d464
Tomasz Grzegorzek
681885e585df02e13b44d465
Pablo Ribalta
681885e585df02e13b44d466
Monika Katariya
681885e585df02e13b44d467
Joey Conway
681885e585df02e13b44d468
Trisha Saar
681885e585df02e13b44d469
Ann Guan
681885e585df02e13b44d46a
Krzysztof Pawelec
681885e585df02e13b44d46b
Shyamala Prayaga
681885e585df02e13b44d46c
Oleksii Kuchaiev
681885e585df02e13b44d46d
Boris Ginsburg
681885e585df02e13b44d46e
Oluwatobi Olabiyi
681885e585df02e13b44d46f
Kari Briski
681885e585df02e13b44d470
Jonathan Cohen
681885e585df02e13b44d471
Bryan Catanzaro
681885e585df02e13b44d472
Jonah Alben
681885e585df02e13b44d473
Yonatan Geifman
681885e585df02e13b44d474
Eric Chung
Llama-Nemotron: Efficient Reasoning Models
We introduce the Llama-Nemotron series of models, an open family of
heterogeneous reasoning models that deliver exceptional reasoning capabilities,
inference efficiency, and an open license for enterprise use. The family comes
in three sizes -- Nano (8B), Super (49B), and Ultra (253B) -- and performs
competitively with state-of-the-art reasoning models such as DeepSeek-R1 while
offering superior inference throughput and memory efficiency. In this report,
we discuss the training procedure for these models, which entails using neural
architecture search from Llama 3 models for accelerated inference, knowledge
distillation, and continued pretraining, followed by a reasoning-focused
post-training stage consisting of two main parts: supervised fine-tuning and
large scale reinforcement learning. Llama-Nemotron models are the first
open-source models to support a dynamic reasoning toggle, allowing users to
switch between standard chat and reasoning modes during inference. To further
support open research and facilitate model development, we provide the
following resources: 1. We release the Llama-Nemotron reasoning models --
LN-Nano, LN-Super, and LN-Ultra -- under the commercially permissive NVIDIA
Open Model License Agreement. 2. We release the complete post-training dataset:
Llama-Nemotron-Post-Training-Dataset. 3. We also release our training
codebases: NeMo, NeMo-Aligner, and Megatron-LM.
Llama-Nemotron models offer exceptional reasoning capabilities, inference efficiency, and open licensing through neural architecture search, knowledge distillation, and reasoning-focused post-training, including supervised fine-tuning and reinforcement learning.
neural architecture search knowledge distillation supervised fine-tuning reinforcement learning dynamic reasoning toggle