title: Llama-Nemotron: Efficient Reasoning Models
summary: We introduce the Llama-Nemotron series of models, an open family of
  heterogeneous reasoning models that deliver exceptional reasoning capabilities,
  inference efficiency, and an open license for enterprise use. The family comes
  in three sizes -- Nano (8B), Super (49B), and Ultra (253B) -- and performs
  competitively with state-of-the-art reasoning models such as DeepSeek-R1 while
  offering superior inference throughput and memory efficiency. In this report,
  we discuss the training procedure for these models, which entails using neural
  architecture search from Llama 3 models for accelerated inference, knowledge
  distillation, and continued pretraining, followed by a reasoning-focused
  post-training stage consisting of two main parts: supervised fine-tuning and
  large scale reinforcement learning. Llama-Nemotron models are the first
  open-source models to support a dynamic reasoning toggle, allowing users to
  switch between standard chat and reasoning modes during inference. To further
  support open research and facilitate model development, we provide the
  following resources: 1. We release the Llama-Nemotron reasoning models --
  LN-Nano, LN-Super, and LN-Ultra -- under the commercially permissive NVIDIA
  Open Model License Agreement. 2. We release the complete post-training dataset:
  Llama-Nemotron-Post-Training-Dataset. 3. We also release our training
  codebases: NeMo, NeMo-Aligner, and Megatron-LM.
paper_authors._id: 681885e585df02e13b44d3f1
paper_authors.name: Akhiad Bercovich
paper_authors._id: 681885e585df02e13b44d3f2
paper_authors.name: Itay Levy
paper_authors._id: 681885e585df02e13b44d3f3
paper_authors.name: Izik Golan
paper_authors._id: 681885e585df02e13b44d3f4
paper_authors.name: Mohammad Dabbah
paper_authors._id: 681885e585df02e13b44d3f5
paper_authors.name: Ran El-Yaniv
paper_authors._id: 681885e585df02e13b44d3f6
paper_authors.name: Omri Puny
paper_authors._id: 681885e585df02e13b44d3f7
paper_authors.name: Ido Galil
paper_authors._id: 681885e585df02e13b44d3f8
paper_authors.name: Zach Moshe
paper_authors._id: 681885e585df02e13b44d3f9
paper_authors.name: Tomer Ronen
paper_authors._id: 681885e585df02e13b44d3fa
paper_authors.name: Najeeb Nabwani
paper_authors._id: 681885e585df02e13b44d3fb
paper_authors.name: Ido Shahaf
paper_authors._id: 681885e585df02e13b44d3fc
paper_authors.name: Oren Tropp
paper_authors._id: 681885e585df02e13b44d3fd
paper_authors.name: Ehud Karpas
paper_authors._id: 681885e585df02e13b44d3fe
paper_authors.name: Ran Zilberstein
paper_authors._id: 681885e585df02e13b44d3ff
paper_authors.name: Jiaqi Zeng
paper_authors._id: 681885e585df02e13b44d400
paper_authors.name: Soumye Singhal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:34:23.674Z
paper_authors._id: 623a5318d2f0273325e008b9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668579465278-623a5318d2f0273325e008b9.jpeg
paper_authors.fullname: Soumye Singhal
paper_authors.type: user
paper_authors.user: soumye
paper_authors._id: 681885e585df02e13b44d401
paper_authors.name: Alexander Bukharin
paper_authors._id: 681885e585df02e13b44d402
paper_authors.name: Yian Zhang
paper_authors._id: 681885e585df02e13b44d403
paper_authors.name: Tugrul Konuk
paper_authors._id: 681885e585df02e13b44d404
paper_authors.name: Gerald Shen
paper_authors._id: 681885e585df02e13b44d405
paper_authors.name: Ameya Sunil Mahabaleshwarkar
paper_authors._id: 681885e585df02e13b44d406
paper_authors.name: Bilal Kartal
paper_authors._id: 681885e585df02e13b44d407
paper_authors.name: Yoshi Suhara
paper_authors._id: 681885e585df02e13b44d408
paper_authors.name: Olivier Delalleau
paper_authors._id: 681885e585df02e13b44d409
paper_authors.name: Zijia Chen
paper_authors._id: 681885e585df02e13b44d40a
paper_authors.name: Zhilin Wang
paper_authors._id: 681885e585df02e13b44d40b
paper_authors.name: David Mosallanezhad
paper_authors._id: 681885e585df02e13b44d40c
paper_authors.name: Adi Renduchintala
paper_authors._id: 681885e585df02e13b44d40d
paper_authors.name: Haifeng Qian
paper_authors._id: 681885e585df02e13b44d40e
paper_authors.name: Dima Rekesh
paper_authors._id: 681885e585df02e13b44d40f
paper_authors.name: Fei Jia
paper_authors._id: 681885e585df02e13b44d410
paper_authors.name: Somshubra Majumdar
paper_authors._id: 681885e585df02e13b44d411
paper_authors.name: Vahid Noroozi
paper_authors._id: 681885e585df02e13b44d412
paper_authors.name: Wasi Uddin Ahmad
paper_authors._id: 681885e585df02e13b44d413
paper_authors.name: Sean Narenthiran
paper_authors._id: 681885e585df02e13b44d414
paper_authors.name: Aleksander Ficek
paper_authors._id: 681885e585df02e13b44d415
paper_authors.name: Mehrzad Samadi
paper_authors._id: 681885e585df02e13b44d416
paper_authors.name: Jocelyn Huang
paper_authors._id: 681885e585df02e13b44d417
paper_authors.name: Siddhartha Jain
paper_authors._id: 681885e585df02e13b44d418
paper_authors.name: Igor Gitman
paper_authors._id: 681885e585df02e13b44d419
paper_authors.name: Ivan Moshkov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:34:29.021Z
paper_authors._id: 65cd2166c52afc14cebc171a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65cd2166c52afc14cebc171a/LhIg07Yaqo0gmqYzGCcTe.png
paper_authors.fullname: Ivan Moshkov
paper_authors.type: user
paper_authors.user: imoshkov
paper_authors._id: 681885e585df02e13b44d41a
paper_authors.name: Wei Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-07T08:09:46.305Z
paper_authors._id: 662fcd0314e87951eaded842
paper_authors.avatarUrl: /avatars/868eccf33126298bd47ecac9da810116.svg
paper_authors.fullname: Wei Du
paper_authors.type: user
paper_authors.user: woshiweidu
paper_authors._id: 681885e585df02e13b44d41b
paper_authors.name: Shubham Toshniwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:34:26.150Z
paper_authors._id: 65c55fbe488606617e9dbabd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65c55fbe488606617e9dbabd/42xMtIIPZPCoF-bYHYMDz.jpeg
paper_authors.fullname: Shubham Toshniwal
paper_authors.type: user
paper_authors.user: stoshniwal
paper_authors._id: 681885e585df02e13b44d41c
paper_authors.name: George Armstrong
paper_authors._id: 681885e585df02e13b44d41d
paper_authors.name: Branislav Kisacanin
paper_authors._id: 681885e585df02e13b44d41e
paper_authors.name: Matvei Novikov
paper_authors._id: 681885e585df02e13b44d41f
paper_authors.name: Daria Gitman
paper_authors._id: 681885e585df02e13b44d420
paper_authors.name: Evelina Bakhturina
paper_authors._id: 681885e585df02e13b44d421
paper_authors.name: Jane Polak Scowcroft
paper_authors._id: 681885e585df02e13b44d422
paper_authors.name: John Kamalu
paper_authors._id: 681885e585df02e13b44d423
paper_authors.name: Dan Su
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:29:15.106Z
paper_authors._id: 67e34615c15564fd97f697ce
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/BNXjsEA5rGn7uFoFuVkB6.png
paper_authors.fullname: Dan Su
paper_authors.type: user
paper_authors.user: sudandandansu1
paper_authors._id: 681885e585df02e13b44d424
paper_authors.name: Kezhi Kong
paper_authors._id: 681885e585df02e13b44d425
paper_authors.name: Markus Kliegl
paper_authors._id: 681885e585df02e13b44d426
paper_authors.name: Rabeeh Karimi
paper_authors._id: 681885e585df02e13b44d427
paper_authors.name: Ying Lin
paper_authors._id: 681885e585df02e13b44d428
paper_authors.name: Sanjeev Satheesh
paper_authors._id: 681885e585df02e13b44d429
paper_authors.name: Jupinder Parmar
paper_authors._id: 681885e585df02e13b44d42a
paper_authors.name: Pritam Gundecha
paper_authors._id: 681885e585df02e13b44d42b
paper_authors.name: Brandon Norick
paper_authors._id: 681885e585df02e13b44d42c
paper_authors.name: Joseph Jennings
paper_authors._id: 681885e585df02e13b44d42d
paper_authors.name: Shrimai Prabhumoye
paper_authors._id: 681885e585df02e13b44d42e
paper_authors.name: Syeda Nahida Akter
paper_authors._id: 681885e585df02e13b44d42f
paper_authors.name: Mostofa Patwary
paper_authors._id: 681885e585df02e13b44d430
paper_authors.name: Abhinav Khattar
paper_authors._id: 681885e585df02e13b44d431
paper_authors.name: Deepak Narayanan
paper_authors._id: 681885e585df02e13b44d432
paper_authors.name: Roger Waleffe
paper_authors._id: 681885e585df02e13b44d433
paper_authors.name: Jimmy Zhang
paper_authors._id: 681885e585df02e13b44d434
paper_authors.name: Bor-Yiing Su
paper_authors._id: 681885e585df02e13b44d435
paper_authors.name: Guyue Huang
paper_authors._id: 681885e585df02e13b44d436
paper_authors.name: Terry Kong
paper_authors._id: 681885e585df02e13b44d437
paper_authors.name: Parth Chadha
paper_authors._id: 681885e585df02e13b44d438
paper_authors.name: Sahil Jain
paper_authors._id: 681885e585df02e13b44d439
paper_authors.name: Christine Harvey
paper_authors._id: 681885e585df02e13b44d43a
paper_authors.name: Elad Segal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-15T10:33:15.041Z
paper_authors._id: 5f5b0efe10b2753d9000c888
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1628140531144-5f5b0efe10b2753d9000c888.jpeg
paper_authors.fullname: Elad Segal
paper_authors.type: user
paper_authors.user: eladsegal
paper_authors._id: 681885e585df02e13b44d43b
paper_authors.name: Jining Huang
paper_authors._id: 681885e585df02e13b44d43c
paper_authors.name: Sergey Kashirsky
paper_authors._id: 681885e585df02e13b44d43d
paper_authors.name: Robert McQueen
paper_authors._id: 681885e585df02e13b44d43e
paper_authors.name: Izzy Putterman
paper_authors._id: 681885e585df02e13b44d43f
paper_authors.name: George Lam
paper_authors._id: 681885e585df02e13b44d440
paper_authors.name: Arun Venkatesan
paper_authors._id: 681885e585df02e13b44d441
paper_authors.name: Sherry Wu
paper_authors._id: 681885e585df02e13b44d442
paper_authors.name: Vinh Nguyen
paper_authors._id: 681885e585df02e13b44d443
paper_authors.name: Manoj Kilaru
paper_authors._id: 681885e585df02e13b44d444
paper_authors.name: Andrew Wang
paper_authors._id: 681885e585df02e13b44d445
paper_authors.name: Anna Warno
paper_authors._id: 681885e585df02e13b44d446
paper_authors.name: Abhilash Somasamudramath
paper_authors._id: 681885e585df02e13b44d447
paper_authors.name: Sandip Bhaskar
paper_authors._id: 681885e585df02e13b44d448
paper_authors.name: Maka Dong
paper_authors._id: 681885e585df02e13b44d449
paper_authors.name: Nave Assaf
paper_authors._id: 681885e585df02e13b44d44a
paper_authors.name: Shahar Mor
paper_authors._id: 681885e585df02e13b44d44b
paper_authors.name: Omer Ullman Argov
paper_authors._id: 681885e585df02e13b44d44c
paper_authors.name: Scot Junkin
paper_authors._id: 681885e585df02e13b44d44d
paper_authors.name: Oleksandr Romanenko
paper_authors._id: 681885e585df02e13b44d44e
paper_authors.name: Pedro Larroy
paper_authors._id: 681885e585df02e13b44d44f
paper_authors.name: Monika Katariya
paper_authors._id: 681885e585df02e13b44d450
paper_authors.name: Marco Rovinelli
paper_authors._id: 681885e585df02e13b44d451
paper_authors.name: Viji Balas
paper_authors._id: 681885e585df02e13b44d452
paper_authors.name: Nicholas Edelman
paper_authors._id: 681885e585df02e13b44d453
paper_authors.name: Anahita Bhiwandiwalla
paper_authors._id: 681885e585df02e13b44d454
paper_authors.name: Muthu Subramaniam
paper_authors._id: 681885e585df02e13b44d455
paper_authors.name: Smita Ithape
paper_authors._id: 681885e585df02e13b44d456
paper_authors.name: Karthik Ramamoorthy
paper_authors._id: 681885e585df02e13b44d457
paper_authors.name: Yuting Wu
paper_authors._id: 681885e585df02e13b44d458
paper_authors.name: Suguna Varshini Velury
paper_authors._id: 681885e585df02e13b44d459
paper_authors.name: Omri Almog
paper_authors._id: 681885e585df02e13b44d45a
paper_authors.name: Joyjit Daw
paper_authors._id: 681885e585df02e13b44d45b
paper_authors.name: Denys Fridman
paper_authors._id: 681885e585df02e13b44d45c
paper_authors.name: Erick Galinkin
paper_authors._id: 681885e585df02e13b44d45d
paper_authors.name: Michael Evans
paper_authors._id: 681885e585df02e13b44d45e
paper_authors.name: Katherine Luna
paper_authors._id: 681885e585df02e13b44d45f
paper_authors.name: Leon Derczynski
paper_authors._id: 681885e585df02e13b44d460
paper_authors.name: Nikki Pope
paper_authors._id: 681885e585df02e13b44d461
paper_authors.name: Eileen Long
paper_authors._id: 681885e585df02e13b44d462
paper_authors.name: Seth Schneider
paper_authors._id: 681885e585df02e13b44d463
paper_authors.name: Guillermo Siman
paper_authors._id: 681885e585df02e13b44d464
paper_authors.name: Tomasz Grzegorzek
paper_authors._id: 681885e585df02e13b44d465
paper_authors.name: Pablo Ribalta
paper_authors._id: 681885e585df02e13b44d466
paper_authors.name: Monika Katariya
paper_authors._id: 681885e585df02e13b44d467
paper_authors.name: Joey Conway
paper_authors._id: 681885e585df02e13b44d468
paper_authors.name: Trisha Saar
paper_authors._id: 681885e585df02e13b44d469
paper_authors.name: Ann Guan
paper_authors._id: 681885e585df02e13b44d46a
paper_authors.name: Krzysztof Pawelec
paper_authors._id: 681885e585df02e13b44d46b
paper_authors.name: Shyamala Prayaga
paper_authors._id: 681885e585df02e13b44d46c
paper_authors.name: Oleksii Kuchaiev
paper_authors._id: 681885e585df02e13b44d46d
paper_authors.name: Boris Ginsburg
paper_authors._id: 681885e585df02e13b44d46e
paper_authors.name: Oluwatobi Olabiyi
paper_authors._id: 681885e585df02e13b44d46f
paper_authors.name: Kari Briski
paper_authors._id: 681885e585df02e13b44d470
paper_authors.name: Jonathan Cohen
paper_authors._id: 681885e585df02e13b44d471
paper_authors.name: Bryan Catanzaro
paper_authors._id: 681885e585df02e13b44d472
paper_authors.name: Jonah Alben
paper_authors._id: 681885e585df02e13b44d473
paper_authors.name: Yonatan Geifman
paper_authors._id: 681885e585df02e13b44d474
paper_authors.name: Eric Chung
paper_title: Llama-Nemotron: Efficient Reasoning Models
paper_summary: We introduce the Llama-Nemotron series of models, an open family of
  heterogeneous reasoning models that deliver exceptional reasoning capabilities,
  inference efficiency, and an open license for enterprise use. The family comes
  in three sizes -- Nano (8B), Super (49B), and Ultra (253B) -- and performs
  competitively with state-of-the-art reasoning models such as DeepSeek-R1 while
  offering superior inference throughput and memory efficiency. In this report,
  we discuss the training procedure for these models, which entails using neural
  architecture search from Llama 3 models for accelerated inference, knowledge
  distillation, and continued pretraining, followed by a reasoning-focused
  post-training stage consisting of two main parts: supervised fine-tuning and
  large scale reinforcement learning. Llama-Nemotron models are the first
  open-source models to support a dynamic reasoning toggle, allowing users to
  switch between standard chat and reasoning modes during inference. To further
  support open research and facilitate model development, we provide the
  following resources: 1. We release the Llama-Nemotron reasoning models --
  LN-Nano, LN-Super, and LN-Ultra -- under the commercially permissive NVIDIA
  Open Model License Agreement. 2. We release the complete post-training dataset:
  Llama-Nemotron-Post-Training-Dataset. 3. We also release our training
  codebases: NeMo, NeMo-Aligner, and Megatron-LM.
paper_ai_summary: Llama-Nemotron models offer exceptional reasoning capabilities, inference efficiency, and open licensing through neural architecture search, knowledge distillation, and reasoning-focused post-training, including supervised fine-tuning and reinforcement learning.
paper_ai_keywords: neural architecture search knowledge distillation supervised fine-tuning reinforcement learning dynamic reasoning toggle