Competitive Programming with Large Reasoning Models
We show that reinforcement learning applied to large language models (LLMs)
significantly boosts performance on complex coding and reasoning tasks.
Additionally, we compare two general-purpose reasoning models - OpenAI o1 and
an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses
hand-engineered inference strategies designed for competing in the 2024
International Olympiad in Informatics (IOI). We competed live at IOI 2024 with
o1-ioi and, using hand-crafted test-time strategies, placed in the 49th
percentile. Under relaxed competition constraints, o1-ioi achieved a gold
medal. However, when evaluating later models such as o3, we find that o3
achieves gold without hand-crafted domain-specific strategies or relaxed
constraints. Our findings show that although specialized pipelines such as
o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model
surpasses those results without relying on hand-crafted inference heuristics.
Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces
rating on par with elite human competitors. Overall, these results indicate
that scaling general-purpose reinforcement learning, rather than relying on
domain-specific techniques, offers a robust path toward state-of-the-art AI in
reasoning domains, such as competitive programming.
67ac1b080686a1e0690741ce
OpenAI
67ac1b080686a1e0690741d0
Ahmed El-Kishky
67ac1b080686a1e0690741d1
Alexander Wei
67ac1b080686a1e0690741d2
Andre Saraiva
67ac1b080686a1e0690741d3
Borys Minaev
67ac1b080686a1e0690741d4
Daniel Selsam
67ac1b080686a1e0690741d5
David Dohan
67ac1b080686a1e0690741d6
Francis Song
67ac1b080686a1e0690741d7
Hunter Lightman
67ac1b080686a1e0690741d8
Ignasi Clavera
67ac1b080686a1e0690741d9
Jakub Pachocki
67ac1b080686a1e0690741da
Jerry Tworek
67ac1b080686a1e0690741db
Lorenz Kuhn
67ac1b080686a1e0690741dc
Lukasz Kaiser
67ac1b080686a1e0690741dd
Mark Chen
67ac1b080686a1e0690741de
Max Schwarzer
67ac1b080686a1e0690741df
Mostafa Rohaninejad
67ac1b080686a1e0690741e0
Nat McAleese
67ac1b080686a1e0690741e1
o3 contributors
67ac1b080686a1e0690741e2
Oleg Mürk
67ac1b080686a1e0690741e3
Rhythm Garg
67ac1b080686a1e0690741e4
Rui Shu
67ac1b080686a1e0690741e5
Szymon Sidor
67ac1b080686a1e0690741e6
Vineet Kosaraju
67ac1b080686a1e0690741e7
Wenda Zhou
Competitive Programming with Large Reasoning Models
We show that reinforcement learning applied to large language models (LLMs)
significantly boosts performance on complex coding and reasoning tasks.
Additionally, we compare two general-purpose reasoning models - OpenAI o1 and
an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses
hand-engineered inference strategies designed for competing in the 2024
International Olympiad in Informatics (IOI). We competed live at IOI 2024 with
o1-ioi and, using hand-crafted test-time strategies, placed in the 49th
percentile. Under relaxed competition constraints, o1-ioi achieved a gold
medal. However, when evaluating later models such as o3, we find that o3
achieves gold without hand-crafted domain-specific strategies or relaxed
constraints. Our findings show that although specialized pipelines such as
o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model
surpasses those results without relying on hand-crafted inference heuristics.
Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces
rating on par with elite human competitors. Overall, these results indicate
that scaling general-purpose reinforcement learning, rather than relying on
domain-specific techniques, offers a robust path toward state-of-the-art AI in
reasoning domains, such as competitive programming.
General-purpose reinforcement learning applied to large language models outperforms domain-specific systems in complex coding and reasoning tasks, achieving top results in competitions without hand-crafted strategies.
reinforcement learning large language models LLMs general-purpose reasoning models domain-specific system hand-engineered inference strategies International Olympiad in Informatics IOI gold medal Codeforces rating competitive programming