content.raw: The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:
content.type: text
content.value: The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:
content.type: new_line
content.type: new_line
content.raw: 🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.
content.type: text
content.value: 🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.
content.type: new_line
content.type: new_line
content.raw: 🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.
content.type: text
content.value: 🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.
content.type: new_line
content.type: new_line
content.raw: 🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.
content.type: text
content.value: 🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.
content.type: new_line
content.type: new_line
content.raw: ⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.
content.type: text
content.value: ⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.
content.type: new_line
content.type: new_line
content.raw: Read the release notes and other resources here 👇
content.type: text
content.value: Read the release notes and other resources here 👇
content.type: new_line
content.raw: Release: 
content.type: text
content.value: Release: 
content.href: https://github.com/huggingface/trl/releases/tag/v0.13.0
content.raw: https://github.com/huggingface/trl/releases/tag/v0.13.0
content.type: link
content.type: new_line
content.raw: Mergekit: 
content.type: text
content.value: Mergekit: 
content.href: https://github.com/arcee-ai/mergekit
content.raw: https://github.com/arcee-ai/mergekit
content.type: link
content.type: new_line
content.raw: Mixture of judges paper: 
content.type: text
content.value: Mixture of judges paper: 
content.label: The Perfect Blend: Redefining RLHF with Mixture of Judges (2409.20370)
content.raw: https://huggingface.co/papers/2409.20370
content.id: 2409.20370
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2409.20370
rawContent: The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:
  
  🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.
  
  🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.
  
  🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.
  
  ⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.
  
  Read the release notes and other resources here 👇
  Release: https://github.com/huggingface/trl/releases/tag/v0.13.0
  Mergekit: https://github.com/arcee-ai/mergekit
  Mixture of judges paper: https://huggingface.co/papers/2409.20370
reactions.reaction: 🔥
reactions.users: prithivMLmods John6666 timrbula TornButter Daemontatox Yukkkop
reactions.reaction: ❤️
reactions.users: MrOvkill