title: Large Language Models Orchestrating Structured Reasoning Achieve Kaggle
    Grandmaster Level
summary: We introduce Agent K v1.0, an end-to-end autonomous data science agent
  designed to automate, optimise, and generalise across diverse data science
  tasks. Fully automated, Agent K v1.0 manages the entire data science life cycle
  by learning from experience. It leverages a highly flexible structured
  reasoning framework to enable it to dynamically process memory in a nested
  structure, effectively learning from accumulated experience stored to handle
  complex reasoning tasks. It optimises long- and short-term memory by
  selectively storing and retrieving key information, guiding future decisions
  based on environmental rewards. This iterative approach allows it to refine
  decisions without fine-tuning or backpropagation, achieving continuous
  improvement through experiential learning. We evaluate our agent's apabilities
  using Kaggle competitions as a case study. Following a fully automated
  protocol, Agent K v1.0 systematically addresses complex and multimodal data
  science tasks, employing Bayesian optimisation for hyperparameter tuning and
  feature engineering. Our new evaluation framework rigorously assesses Agent K
  v1.0's end-to-end capabilities to generate and send submissions starting from a
  Kaggle competition URL. Results demonstrate that Agent K v1.0 achieves a 92.5\%
  success rate across tasks, spanning tabular, computer vision, NLP, and
  multimodal domains. When benchmarking against 5,856 human Kaggle competitors by
  calculating Elo-MMR scores for each, Agent K v1.0 ranks in the top 38\%,
  demonstrating an overall skill level comparable to Expert-level users. Notably,
  its Elo-MMR score falls between the first and third quartiles of scores
  achieved by human Grandmasters. Furthermore, our results indicate that Agent K
  v1.0 has reached a performance level equivalent to Kaggle Grandmaster, with a
  record of 6 gold, 3 silver, and 7 bronze medals, as defined by Kaggle's
  progression system.
paper_authors._id: 672c969016766a76a769a8bc
paper_authors.name: Antoine Grosnit
paper_authors._id: 672c969016766a76a769a8bd
paper_authors.name: Alexandre Maraval
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:33:30.131Z
paper_authors._id: 6320bd0aa023aad6a767c57e
paper_authors.avatarUrl: /avatars/6ac8a8264dec51c4fedf84f040722af1.svg
paper_authors.fullname: Alexandre Maraval
paper_authors.type: user
paper_authors.user: alexmaraval
paper_authors._id: 672c969016766a76a769a8be
paper_authors.name: James Doran
paper_authors._id: 672c969016766a76a769a8bf
paper_authors.name: Giuseppe Paolo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-21T14:43:15.701Z
paper_authors._id: 65e98cd8e19214e9d151f29e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65e98cd8e19214e9d151f29e/XjQzoVgKVzv8AZBWFQnHz.jpeg
paper_authors.fullname: Giuseppe Paolo
paper_authors.type: user
paper_authors.user: GPaolo
paper_authors._id: 672c969016766a76a769a8c0
paper_authors.name: Albert Thomas
paper_authors._id: 672c969016766a76a769a8c1
paper_authors.name: Refinath Shahul Hameed Nabeezath Beevi
paper_authors._id: 672c969016766a76a769a8c2
paper_authors.name: Jonas Gonzalez
paper_authors._id: 672c969016766a76a769a8c3
paper_authors.name: Khyati Khandelwal
paper_authors._id: 672c969016766a76a769a8c4
paper_authors.name: Ignacio Iacobacci
paper_authors._id: 672c969016766a76a769a8c5
paper_authors.name: Abdelhakim Benechehab
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:36:30.657Z
paper_authors._id: 621d59ebd3df05d67132e8d9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/621d59ebd3df05d67132e8d9/0gPfPTRKKnz5kq0InTqm5.jpeg
paper_authors.fullname: Abdelhakim Benechehab
paper_authors.type: user
paper_authors.user: abenechehab
paper_authors._id: 672c969016766a76a769a8c6
paper_authors.name: Hamza Cherkaoui
paper_authors._id: 672c969016766a76a769a8c7
paper_authors.name: Youssef Attia El-Hili
paper_authors._id: 672c969016766a76a769a8c8
paper_authors.name: Kun Shao
paper_authors._id: 672c969016766a76a769a8c9
paper_authors.name: Jianye Hao
paper_authors._id: 672c969016766a76a769a8ca
paper_authors.name: Jun Yao
paper_authors._id: 672c969016766a76a769a8cb
paper_authors.name: Balazs Kegl
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:33:27.140Z
paper_authors._id: 672cb69f250205b317235571
paper_authors.avatarUrl: /avatars/37afc1506b15a2f9c37e3e8769142580.svg
paper_authors.fullname: Balazs Kegl
paper_authors.type: user
paper_authors.user: balazskegl
paper_authors._id: 672c969016766a76a769a8cc
paper_authors.name: Haitham Bou-Ammar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-30T19:37:48.373Z
paper_authors._id: 631c375768f7da9ad2496bf6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631c375768f7da9ad2496bf6/1sDOoecA6e1v_hn_VAgUq.jpeg
paper_authors.fullname: Haitham Bou Ammar
paper_authors.type: user
paper_authors.user: hba123
paper_authors._id: 672c969016766a76a769a8cd
paper_authors.name: Jun Wang
paper_title: Large Language Models Orchestrating Structured Reasoning Achieve Kaggle
    Grandmaster Level
paper_summary: We introduce Agent K v1.0, an end-to-end autonomous data science agent
  designed to automate, optimise, and generalise across diverse data science
  tasks. Fully automated, Agent K v1.0 manages the entire data science life cycle
  by learning from experience. It leverages a highly flexible structured
  reasoning framework to enable it to dynamically process memory in a nested
  structure, effectively learning from accumulated experience stored to handle
  complex reasoning tasks. It optimises long- and short-term memory by
  selectively storing and retrieving key information, guiding future decisions
  based on environmental rewards. This iterative approach allows it to refine
  decisions without fine-tuning or backpropagation, achieving continuous
  improvement through experiential learning. We evaluate our agent's apabilities
  using Kaggle competitions as a case study. Following a fully automated
  protocol, Agent K v1.0 systematically addresses complex and multimodal data
  science tasks, employing Bayesian optimisation for hyperparameter tuning and
  feature engineering. Our new evaluation framework rigorously assesses Agent K
  v1.0's end-to-end capabilities to generate and send submissions starting from a
  Kaggle competition URL. Results demonstrate that Agent K v1.0 achieves a 92.5\%
  success rate across tasks, spanning tabular, computer vision, NLP, and
  multimodal domains. When benchmarking against 5,856 human Kaggle competitors by
  calculating Elo-MMR scores for each, Agent K v1.0 ranks in the top 38\%,
  demonstrating an overall skill level comparable to Expert-level users. Notably,
  its Elo-MMR score falls between the first and third quartiles of scores
  achieved by human Grandmasters. Furthermore, our results indicate that Agent K
  v1.0 has reached a performance level equivalent to Kaggle Grandmaster, with a
  record of 6 gold, 3 silver, and 7 bronze medals, as defined by Kaggle's
  progression system.
paper_ai_summary: Agent K v1.0 automates data science tasks across various domains using structured reasoning and Bayesian optimization, achieving performance comparable to Expert-level Kaggle users.
paper_ai_keywords: structured reasoning memory nested structure Bayesian optimisation feature engineering Elo-MMR scores Kaggle competitions Grandmaster