MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine
  Learning Engineering
We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement
learning, evaluating, and improving autonomous large language model (LLM)
agents in iterative machine learning engineering (MLE) workflows. Unlike
existing benchmarks that primarily rely on static datasets or single-attempt
evaluations, MLE-Dojo provides an interactive environment enabling agents to
iteratively experiment, debug, and refine solutions through structured feedback
loops. Built upon 200+ real-world Kaggle challenges, MLE-Dojo covers diverse,
open-ended MLE tasks carefully curated to reflect realistic engineering
scenarios such as data processing, architecture search, hyperparameter tuning,
and code debugging. Its fully executable environment supports comprehensive
agent training via both supervised fine-tuning and reinforcement learning,
facilitating iterative experimentation, realistic data sampling, and real-time
outcome verification. Extensive evaluations of eight frontier LLMs reveal that
while current models achieve meaningful iterative improvements, they still
exhibit significant limitations in autonomously generating long-horizon
solutions and efficiently resolving complex errors. Furthermore, MLE-Dojo's
flexible and extensible architecture seamlessly integrates diverse data
sources, tools, and evaluation protocols, uniquely enabling model-based agent
tuning and promoting interoperability, scalability, and reproducibility. We
open-source our framework and benchmarks to foster community-driven innovation
towards next-generation MLE agents.
6822b3c8c10ac9c466c63e01
Rushi Qiang
claimed_verified
2025-05-14T07:35:59.704Z
6466e31a14e059dde8bbe4be
https://cdn-avatars.huggingface.co/v1/production/uploads/6466e31a14e059dde8bbe4be/D_DWGWEkhOnFdbbdCNP3N.jpeg
Rushi Qiang
user
Jerrycool
6822b3c8c10ac9c466c63e02
Yuchen Zhuang
admin_assigned
2025-05-16T08:17:45.935Z
6471bddd609ae9f56368f132
https://cdn-avatars.huggingface.co/v1/production/uploads/6471bddd609ae9f56368f132/G91Q4iCGN2dy3oMaz-LrO.jpeg
Yuchen Zhuang
user
yczhuang
6822b3c8c10ac9c466c63e03
Yinghao Li
admin_assigned
2025-05-16T08:17:54.487Z
68198e34612ca40b67abbf18
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Gj9eV6k3CcKJinTePC2CV.png
Yinghao Li
user
yinghaoli-yh
6822b3c8c10ac9c466c63e04
Dingu Sagar V K
6822b3c8c10ac9c466c63e05
Rongzhi Zhang
claimed_verified
2025-05-16T07:12:47.628Z
644a2c4e9a1c5faef7a5dbd8
/avatars/fbbbc1347f8e423b2477e2506fdb43d9.svg
Rongzhi Zhang
user
Solute
6822b3c8c10ac9c466c63e06
Changhao Li
claimed_verified
2025-07-23T08:37:49.162Z
660026d9573abbcdb975aa13
https://cdn-avatars.huggingface.co/v1/production/uploads/660026d9573abbcdb975aa13/XktfFxTMC_d1xJ4pDJLum.jpeg
Changhao
user
lichangh20
6822b3c8c10ac9c466c63e07
Ian Shu-Hei Wong
6822b3c8c10ac9c466c63e08
Sherry Yang
6822b3c8c10ac9c466c63e09
Percy Liang
admin_assigned
2025-05-16T08:18:20.107Z
6409651b9e9f790c905b2335
/avatars/1fb8c80b60f21f65a0a027319101f236.svg
Percy Liang
user
percyliang
6822b3c8c10ac9c466c63e0a
Chao Zhang
6822b3c8c10ac9c466c63e0b
Bo Dai
MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine
  Learning Engineering
We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement
learning, evaluating, and improving autonomous large language model (LLM)
agents in iterative machine learning engineering (MLE) workflows. Unlike
existing benchmarks that primarily rely on static datasets or single-attempt
evaluations, MLE-Dojo provides an interactive environment enabling agents to
iteratively experiment, debug, and refine solutions through structured feedback
loops. Built upon 200+ real-world Kaggle challenges, MLE-Dojo covers diverse,
open-ended MLE tasks carefully curated to reflect realistic engineering
scenarios such as data processing, architecture search, hyperparameter tuning,
and code debugging. Its fully executable environment supports comprehensive
agent training via both supervised fine-tuning and reinforcement learning,
facilitating iterative experimentation, realistic data sampling, and real-time
outcome verification. Extensive evaluations of eight frontier LLMs reveal that
while current models achieve meaningful iterative improvements, they still
exhibit significant limitations in autonomously generating long-horizon
solutions and efficiently resolving complex errors. Furthermore, MLE-Dojo's
flexible and extensible architecture seamlessly integrates diverse data
sources, tools, and evaluation protocols, uniquely enabling model-based agent
tuning and promoting interoperability, scalability, and reproducibility. We
open-source our framework and benchmarks to foster community-driven innovation
towards next-generation MLE agents.
MLE-Dojo provides an interactive framework for reinforcement learning and iterative improvement of large language model agents in real-world machine learning engineering tasks.
MLE-Dojo reinforcement learning autonomous large language model agents iterative machine learning engineering Kaggle challenges data processing architecture search hyperparameter tuning code debugging supervised fine-tuning model-based agent tuning interoperability scalability reproducibility