title: Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement
    Fine-Tuning of Large Language Models
summary: Trinity-RFT is a general-purpose, flexible and scalable framework designed
  for reinforcement fine-tuning (RFT) of large language models. It is built with
  a decoupled design, consisting of (1) an RFT-core that unifies and generalizes
  synchronous/asynchronous, on-policy/off-policy, and online/offline modes of
  RFT, (2) seamless integration for agent-environment interaction with high
  efficiency and robustness, and (3) systematic data pipelines optimized for RFT.
  Trinity-RFT can be easily adapted for diverse application scenarios, and serves
  as a unified platform for exploring advanced reinforcement learning paradigms.
  This technical report outlines the vision, features, design and implementations
  of Trinity-RFT, accompanied by extensive examples demonstrating the utility and
  user-friendliness of the proposed framework.
paper_authors._id: 6833ce1bd5c438959f750d57
paper_authors.name: Xuchen Pan
paper_authors._id: 6833ce1bd5c438959f750d58
paper_authors.name: Yanxi Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:44:36.860Z
paper_authors._id: 6576f9f4654561a1b345610b
paper_authors.avatarUrl: /avatars/f801f551640caa70368fcc26a0f51d27.svg
paper_authors.fullname: Yanxi Chen
paper_authors.type: user
paper_authors.user: yanxi-chen
paper_authors._id: 6833ce1bd5c438959f750d59
paper_authors.name: Yushuo Chen
paper_authors._id: 6833ce1bd5c438959f750d5a
paper_authors.name: Yuchang Sun
paper_authors._id: 6833ce1bd5c438959f750d5b
paper_authors.name: Daoyuan Chen
paper_authors._id: 6833ce1bd5c438959f750d5c
paper_authors.name: Wenhao Zhang
paper_authors._id: 6833ce1bd5c438959f750d5d
paper_authors.name: Yuexiang Xie
paper_authors._id: 6833ce1bd5c438959f750d5e
paper_authors.name: Yilun Huang
paper_authors._id: 6833ce1bd5c438959f750d5f
paper_authors.name: Yilei Zhang
paper_authors._id: 6833ce1bd5c438959f750d60
paper_authors.name: Dawei Gao
paper_authors._id: 6833ce1bd5c438959f750d61
paper_authors.name: Yaliang Li
paper_authors._id: 6833ce1bd5c438959f750d62
paper_authors.name: Bolin Ding
paper_authors._id: 6833ce1bd5c438959f750d63
paper_authors.name: Jingren Zhou
paper_title: Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement
    Fine-Tuning of Large Language Models
paper_summary: Trinity-RFT is a general-purpose, flexible and scalable framework designed
  for reinforcement fine-tuning (RFT) of large language models. It is built with
  a decoupled design, consisting of (1) an RFT-core that unifies and generalizes
  synchronous/asynchronous, on-policy/off-policy, and online/offline modes of
  RFT, (2) seamless integration for agent-environment interaction with high
  efficiency and robustness, and (3) systematic data pipelines optimized for RFT.
  Trinity-RFT can be easily adapted for diverse application scenarios, and serves
  as a unified platform for exploring advanced reinforcement learning paradigms.
  This technical report outlines the vision, features, design and implementations
  of Trinity-RFT, accompanied by extensive examples demonstrating the utility and
  user-friendliness of the proposed framework.
paper_ai_summary: Trinity-RFT is a flexible and scalable framework for reinforcement fine-tuning of large language models, supporting various interaction modes and data pipelines.
paper_ai_keywords: reinforcement fine-tuning RFT-core synchronous/asynchronous on-policy/off-policy online/offline agent-environment interaction data pipelines reinforcement learning paradigms