Large Language Models Orchestrating Structured Reasoning Achieve Kaggle
  Grandmaster Level
We introduce Agent K v1.0, an end-to-end autonomous data science agent
designed to automate, optimise, and generalise across diverse data science
tasks. Fully automated, Agent K v1.0 manages the entire data science life cycle
by learning from experience. It leverages a highly flexible structured
reasoning framework to enable it to dynamically process memory in a nested
structure, effectively learning from accumulated experience stored to handle
complex reasoning tasks. It optimises long- and short-term memory by
selectively storing and retrieving key information, guiding future decisions
based on environmental rewards. This iterative approach allows it to refine
decisions without fine-tuning or backpropagation, achieving continuous
improvement through experiential learning. We evaluate our agent's apabilities
using Kaggle competitions as a case study. Following a fully automated
protocol, Agent K v1.0 systematically addresses complex and multimodal data
science tasks, employing Bayesian optimisation for hyperparameter tuning and
feature engineering. Our new evaluation framework rigorously assesses Agent K
v1.0's end-to-end capabilities to generate and send submissions starting from a
Kaggle competition URL. Results demonstrate that Agent K v1.0 achieves a 92.5\%
success rate across tasks, spanning tabular, computer vision, NLP, and
multimodal domains. When benchmarking against 5,856 human Kaggle competitors by
calculating Elo-MMR scores for each, Agent K v1.0 ranks in the top 38\%,
demonstrating an overall skill level comparable to Expert-level users. Notably,
its Elo-MMR score falls between the first and third quartiles of scores
achieved by human Grandmasters. Furthermore, our results indicate that Agent K
v1.0 has reached a performance level equivalent to Kaggle Grandmaster, with a
record of 6 gold, 3 silver, and 7 bronze medals, as defined by Kaggle's
progression system.
672c969016766a76a769a8bc
Antoine Grosnit
672c969016766a76a769a8bd
Alexandre Maraval
claimed_verified
2024-11-15T09:33:30.131Z
6320bd0aa023aad6a767c57e
/avatars/6ac8a8264dec51c4fedf84f040722af1.svg
Alexandre Maraval
user
alexmaraval
672c969016766a76a769a8be
James Doran
672c969016766a76a769a8bf
Giuseppe Paolo
claimed_verified
2025-02-21T14:43:15.701Z
65e98cd8e19214e9d151f29e
https://cdn-avatars.huggingface.co/v1/production/uploads/65e98cd8e19214e9d151f29e/XjQzoVgKVzv8AZBWFQnHz.jpeg
Giuseppe Paolo
user
GPaolo
672c969016766a76a769a8c0
Albert Thomas
672c969016766a76a769a8c1
Refinath Shahul Hameed Nabeezath Beevi
672c969016766a76a769a8c2
Jonas Gonzalez
672c969016766a76a769a8c3
Khyati Khandelwal
672c969016766a76a769a8c4
Ignacio Iacobacci
672c969016766a76a769a8c5
Abdelhakim Benechehab
claimed_verified
2025-02-18T09:36:30.657Z
621d59ebd3df05d67132e8d9
https://cdn-avatars.huggingface.co/v1/production/uploads/621d59ebd3df05d67132e8d9/0gPfPTRKKnz5kq0InTqm5.jpeg
Abdelhakim Benechehab
user
abenechehab
672c969016766a76a769a8c6
Hamza Cherkaoui
672c969016766a76a769a8c7
Youssef Attia El-Hili
672c969016766a76a769a8c8
Kun Shao
672c969016766a76a769a8c9
Jianye Hao
672c969016766a76a769a8ca
Jun Yao
672c969016766a76a769a8cb
Balazs Kegl
claimed_verified
2024-11-15T09:33:27.140Z
672cb69f250205b317235571
/avatars/37afc1506b15a2f9c37e3e8769142580.svg
Balazs Kegl
user
balazskegl
672c969016766a76a769a8cc
Haitham Bou-Ammar
claimed_verified
2024-12-30T19:37:48.373Z
631c375768f7da9ad2496bf6
https://cdn-avatars.huggingface.co/v1/production/uploads/631c375768f7da9ad2496bf6/1sDOoecA6e1v_hn_VAgUq.jpeg
Haitham Bou Ammar
user
hba123
672c969016766a76a769a8cd
Jun Wang
Large Language Models Orchestrating Structured Reasoning Achieve Kaggle
  Grandmaster Level
We introduce Agent K v1.0, an end-to-end autonomous data science agent
designed to automate, optimise, and generalise across diverse data science
tasks. Fully automated, Agent K v1.0 manages the entire data science life cycle
by learning from experience. It leverages a highly flexible structured
reasoning framework to enable it to dynamically process memory in a nested
structure, effectively learning from accumulated experience stored to handle
complex reasoning tasks. It optimises long- and short-term memory by
selectively storing and retrieving key information, guiding future decisions
based on environmental rewards. This iterative approach allows it to refine
decisions without fine-tuning or backpropagation, achieving continuous
improvement through experiential learning. We evaluate our agent's apabilities
using Kaggle competitions as a case study. Following a fully automated
protocol, Agent K v1.0 systematically addresses complex and multimodal data
science tasks, employing Bayesian optimisation for hyperparameter tuning and
feature engineering. Our new evaluation framework rigorously assesses Agent K
v1.0's end-to-end capabilities to generate and send submissions starting from a
Kaggle competition URL. Results demonstrate that Agent K v1.0 achieves a 92.5\%
success rate across tasks, spanning tabular, computer vision, NLP, and
multimodal domains. When benchmarking against 5,856 human Kaggle competitors by
calculating Elo-MMR scores for each, Agent K v1.0 ranks in the top 38\%,
demonstrating an overall skill level comparable to Expert-level users. Notably,
its Elo-MMR score falls between the first and third quartiles of scores
achieved by human Grandmasters. Furthermore, our results indicate that Agent K
v1.0 has reached a performance level equivalent to Kaggle Grandmaster, with a
record of 6 gold, 3 silver, and 7 bronze medals, as defined by Kaggle's
progression system.
Agent K v1.0 automates data science tasks across various domains using structured reasoning and Bayesian optimization, achieving performance comparable to Expert-level Kaggle users.
structured reasoning memory nested structure Bayesian optimisation feature engineering Elo-MMR scores Kaggle competitions Grandmaster