title: MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine
    Learning Engineering
summary: We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement
  learning, evaluating, and improving autonomous large language model (LLM)
  agents in iterative machine learning engineering (MLE) workflows. Unlike
  existing benchmarks that primarily rely on static datasets or single-attempt
  evaluations, MLE-Dojo provides an interactive environment enabling agents to
  iteratively experiment, debug, and refine solutions through structured feedback
  loops. Built upon 200+ real-world Kaggle challenges, MLE-Dojo covers diverse,
  open-ended MLE tasks carefully curated to reflect realistic engineering
  scenarios such as data processing, architecture search, hyperparameter tuning,
  and code debugging. Its fully executable environment supports comprehensive
  agent training via both supervised fine-tuning and reinforcement learning,
  facilitating iterative experimentation, realistic data sampling, and real-time
  outcome verification. Extensive evaluations of eight frontier LLMs reveal that
  while current models achieve meaningful iterative improvements, they still
  exhibit significant limitations in autonomously generating long-horizon
  solutions and efficiently resolving complex errors. Furthermore, MLE-Dojo's
  flexible and extensible architecture seamlessly integrates diverse data
  sources, tools, and evaluation protocols, uniquely enabling model-based agent
  tuning and promoting interoperability, scalability, and reproducibility. We
  open-source our framework and benchmarks to foster community-driven innovation
  towards next-generation MLE agents.
paper_authors._id: 6822b3c8c10ac9c466c63e01
paper_authors.name: Rushi Qiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-14T07:35:59.704Z
paper_authors._id: 6466e31a14e059dde8bbe4be
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6466e31a14e059dde8bbe4be/D_DWGWEkhOnFdbbdCNP3N.jpeg
paper_authors.fullname: Rushi Qiang
paper_authors.type: user
paper_authors.user: Jerrycool
paper_authors._id: 6822b3c8c10ac9c466c63e02
paper_authors.name: Yuchen Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:17:45.935Z
paper_authors._id: 6471bddd609ae9f56368f132
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6471bddd609ae9f56368f132/G91Q4iCGN2dy3oMaz-LrO.jpeg
paper_authors.fullname: Yuchen Zhuang
paper_authors.type: user
paper_authors.user: yczhuang
paper_authors._id: 6822b3c8c10ac9c466c63e03
paper_authors.name: Yinghao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:17:54.487Z
paper_authors._id: 68198e34612ca40b67abbf18
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Gj9eV6k3CcKJinTePC2CV.png
paper_authors.fullname: Yinghao Li
paper_authors.type: user
paper_authors.user: yinghaoli-yh
paper_authors._id: 6822b3c8c10ac9c466c63e04
paper_authors.name: Dingu Sagar V K
paper_authors._id: 6822b3c8c10ac9c466c63e05
paper_authors.name: Rongzhi Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-16T07:12:47.628Z
paper_authors._id: 644a2c4e9a1c5faef7a5dbd8
paper_authors.avatarUrl: /avatars/fbbbc1347f8e423b2477e2506fdb43d9.svg
paper_authors.fullname: Rongzhi Zhang
paper_authors.type: user
paper_authors.user: Solute
paper_authors._id: 6822b3c8c10ac9c466c63e06
paper_authors.name: Changhao Li
paper_authors._id: 6822b3c8c10ac9c466c63e07
paper_authors.name: Ian Shu-Hei Wong
paper_authors._id: 6822b3c8c10ac9c466c63e08
paper_authors.name: Sherry Yang
paper_authors._id: 6822b3c8c10ac9c466c63e09
paper_authors.name: Percy Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:18:20.107Z
paper_authors._id: 6409651b9e9f790c905b2335
paper_authors.avatarUrl: /avatars/1fb8c80b60f21f65a0a027319101f236.svg
paper_authors.fullname: Percy Liang
paper_authors.type: user
paper_authors.user: percyliang
paper_authors._id: 6822b3c8c10ac9c466c63e0a
paper_authors.name: Chao Zhang
paper_authors._id: 6822b3c8c10ac9c466c63e0b
paper_authors.name: Bo Dai
paper_title: MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine
    Learning Engineering
paper_summary: We introduce MLE-Dojo, a Gym-style framework for systematically reinforcement
  learning, evaluating, and improving autonomous large language model (LLM)
  agents in iterative machine learning engineering (MLE) workflows. Unlike
  existing benchmarks that primarily rely on static datasets or single-attempt
  evaluations, MLE-Dojo provides an interactive environment enabling agents to
  iteratively experiment, debug, and refine solutions through structured feedback
  loops. Built upon 200+ real-world Kaggle challenges, MLE-Dojo covers diverse,
  open-ended MLE tasks carefully curated to reflect realistic engineering
  scenarios such as data processing, architecture search, hyperparameter tuning,
  and code debugging. Its fully executable environment supports comprehensive
  agent training via both supervised fine-tuning and reinforcement learning,
  facilitating iterative experimentation, realistic data sampling, and real-time
  outcome verification. Extensive evaluations of eight frontier LLMs reveal that
  while current models achieve meaningful iterative improvements, they still
  exhibit significant limitations in autonomously generating long-horizon
  solutions and efficiently resolving complex errors. Furthermore, MLE-Dojo's
  flexible and extensible architecture seamlessly integrates diverse data
  sources, tools, and evaluation protocols, uniquely enabling model-based agent
  tuning and promoting interoperability, scalability, and reproducibility. We
  open-source our framework and benchmarks to foster community-driven innovation
  towards next-generation MLE agents.
paper_ai_summary: MLE-Dojo provides an interactive framework for reinforcement learning and iterative improvement of large language model agents in real-world machine learning engineering tasks.
paper_ai_keywords: MLE-Dojo reinforcement learning autonomous large language model agents iterative machine learning engineering Kaggle challenges data processing architecture search hyperparameter tuning code debugging supervised fine-tuning model-based agent tuning interoperability scalability reproducibility