agents-course (Hugging Face Agents Course)

The latest piece by @MiniMax-AI is a must-read.

It tries to break the impossible triangle of agent RL: throughput × stability × flexibility.

A lot to learn here, go read it 🫵
https://huggingface.co/blog/MiniMax-AI/forge-scalable-agent-rl-framework-and-algorithm

sergiopaniego

updated a model 8 days ago

agents-course/notebooks

Updated 8 days ago • 519

sergiopaniego

posted an update 9 days ago

Post

395

if you're looking for a good first issue to get your open-source journey started, you could contribute to this TRL issue by documenting one impactful paper in the docs

we have a broad list to cover!! 🧐

https://github.com/huggingface/trl/issues/4407

sergiopaniego

posted an update 20 days ago

Post

466

Meet the Post-Training Toolkit (PTT), which easily integrates with TRL via a single callback, by Aditya Challapally ( @microsoft ):

🔍 Detects training issues early
🛠 Lets you intervene safely
📊 Keeps long training runs stable, auditable & efficient

Microsoft blog: https://devblogs.microsoft.com/engineering-at-microsoft/diagnosing-instability-in-production-scale-agent-rl/

Integration guide: https://huggingface.co/docs/trl/main/en/ptt_integration

Code: https://github.com/microsoft/post-training-toolkit

sergiopaniego

posted an update 21 days ago

Post

2551

New TRL + OpenEnv example! 💥

Fine tune an LLM for playing Sudoku using an RL env via OpenEnv

Includes a script that runs on 1 or multiple GPUs with vLLM, plus a Colab-ready notebook.

Enjoy!

Notebook: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/openenv_sudoku_grpo.ipynb

Script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/sudoku.py

1 reply

·

sergiopaniego

posted an update 23 days ago

Post

2161

Date idea: read the entire Transformers v5.0.0 release notes

Officially stable now: https://github.com/huggingface/transformers/releases/tag/v5.0.0

1 reply

·

sergiopaniego

posted an update about 1 month ago

Post

1622

FunctionGemma Tuning Lab is a new no-code tool by @google that lets you fine-tune a model directly from the browser, with no coding knowledge required, using TRL behind the scenes.

blog: https://developers.googleblog.com/a-guide-to-fine-tuning-functiongemma/

try it out: google/functiongemma-tuning-lab

This example builds on a more advanced one for learning fine-tuning with SFT using TRL: https://ai.google.dev/gemma/docs/functiongemma/finetuning-with-functiongemma

1 reply

·

sergiopaniego

posted an update about 1 month ago

Post

835

TRL v0.27.0 is out!! 🥳

It includes GDPO, the latest variant of GRPO for multi-reward RL ✨
GDPO decouples reward normalization to avoid reward collapse and improve per-reward convergence — developed by
@sliuau @SimonX et al.

Explore the paper: GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization (2601.05242)

Explore the full set of changes here:
https://github.com/huggingface/trl/releases/tag/v0.27.0

sergiopaniego

posted an update about 1 month ago

Post

3026

New REPL environment in OpenEnv available! ✨
Used in the Recursive Language Models (RLM) paper by Alex Zhang.

Ready for inference & post-training using trajectories. Handles long contexts:

> Run Python code in a sandbox
> Make recursive calls to LMs
> Explore data programmatically
> Return final result

Docs: https://meta-pytorch.org/OpenEnv/environments/repl/
Inference script: https://github.com/meta-pytorch/OpenEnv/blob/main/examples/repl_oolong_simple.py

sergiopaniego

posted an update about 1 month ago

Post

550

Recursive Language Models (RLM) is a new interface for LLMs with cool ideas by Alex Zhang!

⚠️ LLMs struggle with long prompts → attention overload & lost info
🔄 RLMs inspect, split & call themselves on chunks, then aggregate results
✅ Handles millions of tokens, reduces noise, improves reasoning
💡 System prompt guides recursion
🎯 RLM trajectories can be used for RL training or distillation (OpenEnv+TRL!!)

We're adding it to OpenEnv (with Kashif Rasul): https://github.com/meta-pytorch/OpenEnv/pull/282

More resources:

> Paper: Recursive Language Models (2512.24601)
> Paper blog: https://alexzhang13.github.io/blog/2025/rlm/
> RLM repo: https://github.com/alexzhang13/rlm

2 replies

·

sergiopaniego

posted an update about 1 month ago

Post

2277

New GRPO + TRL free Colab notebook out! 🔥

Fine-tune 7B+ models on T4 GPUs thanks to a ton of memory optimizations for GRPO

7B model uses only 9.2 GB VRAM (~7× reduction) 🤯

Try the notebook here 👉 https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_trl_lora_qlora.ipynb

pcuenq

posted an update about 1 month ago

Post

3399

👉 What happened in AI in 2025? 👈

We prepared the 2025 version of the HF AI Timeline Grid, highlighting open vs API-based model releases, and allowing you to browse and filter by access, modality, and release type!

Play with it here:
2025-ai-timeline/2025-ai-timeline

Here's my personal quarterly TL;DR:

1️⃣ Q1 — Learning to Reason
Deepseek not only releases a top-notch reasoning model, but shows how to train them and compete with closed frontier models. OpenAI debuts Deep Research.

Significant milestones: DeepSeek R1 & R1-Zero, Qwen 2.5 VL, OpenAI Deep Research, Gemini 2.5 Pro (experimental)

2️⃣ Q2 — Multimodality and Coding
More LLMs embrace multimodality by default, and there's a surge in coding agents. Strong vision, audio, and generative models emerge.

Significant milestones: Llama 4, Qwen 3, Imagen 4, OpenAI Codex, Google Jules, Claude 4

3️⃣ Q3 — "Gold" rush, OpenAI opens up, the community goes bananas
Flagship models get gold in Math olympiads and hard benchmarks. OpenAI releases strong open source models and Google releases the much anticipated nano-banana for image generation and editing. Agentic workflows become commonplace.

Significant milestones: Gemini and OpenAI IMO Gold, gpt-oss, Gemini 2.5 Flash Image, Grok 4, Claude Sonnet 4.5

4️⃣ Q4 — Mistral returns, leaderboard hill-climbing
Mistral is back with updated model families. All labs release impressive models to wrap up the year!

Significant milestones: Claude Opus 4.5, DeepSeek Math V2, FLUX 2, GPT 5.1, Kimi K2 Thinking, Nano Banana Pro, GLM 4.7, Gemini 3, Mistral 3, MiniMax M2.1 🤯

Credits
🙏 NHLOCAL for the source data https://github.com/NHLOCAL/AiTimeline

🫡 @reach-vb for the original idea, design and recipe

🙌 @ariG23498 and yours truly for compiling and verifying the 2025 edition

🥳 Here's to 2026, wishing it becomes the best year ever for open releases and on-device-first use-cases! 🥂

2 replies

·

sergiopaniego

posted an update about 2 months ago

Post

2613

The list of hands-on notebooks (some beginner-friendly!) to get started with fine-tuning using TRL keeps growing!!

• SFT
• GRPO
• Tool calling & agents
• RL environments with OpenEnv
• LLMs and VLMs
✨ Many run on FREE Colab, making it super easy to get started fast!

https://github.com/huggingface/trl/tree/main/examples/notebooks

sergiopaniego

posted an update about 2 months ago

Post

505

As the year comes to an end, it’s a good moment to catch up on some of the best long-form pieces published by the @huggingface team.

I’ve gathered them all here if you want to read or save them for later:
https://huggingface.co/collections/sergiopaniego/research-and-long-form-blog-posts

sergiopaniego

posted an update about 2 months ago

Post

2328

This super detailed tutorial by @Paulescu is pure gold 🪙 "Fine-tuning a Small Language Model for browser control with GRPO and OpenEnv"

LFM2-350M ( @LiquidAI ) + BrowserGym (OpenEnv) + GRPO (TRL) for learning browser control 🤝

https://paulabartabajo.substack.com/p/fine-tuning-lfm2-350m-for-browser

Hugging Face Agents Course

AI & ML interests

Recent Activity

agents-course/unit4-students-scores

agents-course/final-certificates

agents-course/course-certificates-of-excellence

agents-course/certificates

agents-course/notebooks

AI & ML interests

Recent Activity

Team members 9

agents-course's activity