Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
Large language models are typically adapted to downstream tasks through
supervised fine-tuning on domain-specific data. While standard fine-tuning
focuses on minimizing generation loss to optimize model parameters, we take a
deeper step by retaining and leveraging the model's own learning signals,
analogous to how human learners reflect on past mistakes to improve future
performance. We first introduce the concept of Mistake Log to systematically
track the model's learning behavior and recurring errors throughout
fine-tuning. Treating the original transformer-based model as the Pilot, we
correspondingly design a Copilot model to refine the Pilot's inference
performance via logits rectification. We name the overall Pilot-Copilot
framework the Transformer Copilot, which introduces (i) a novel Copilot model
design, (ii) a joint training paradigm where the Copilot continuously learns
from the evolving Mistake Log alongside the Pilot, and (iii) a fused inference
paradigm where the Copilot rectifies the Pilot's logits for enhanced
generation. We provide both theoretical and empirical analyses on our new
learning framework. Experiments on 12 benchmarks spanning commonsense,
arithmetic, and recommendation tasks demonstrate that Transformer Copilot
consistently improves performance by up to 34.5%, while introducing marginal
computational overhead to Pilot models and exhibiting strong scalability and
transferability.
6833d08edf7cbb5c087a8bf1
Jiaru Zou
extracted_confirmed
2025-05-26T02:41:41.249Z
65c288280aa2d53135734a42
https://cdn-avatars.huggingface.co/v1/production/uploads/65c288280aa2d53135734a42/5WHmau52EaRS01TOMI3Qg.jpeg
Jiaru Zou
user
jiaruz2
6833d08edf7cbb5c087a8bf2
Yikun Ban
6833d08edf7cbb5c087a8bf3
Zihao Li
6833d08edf7cbb5c087a8bf4
Yunzhe Qi
6833d08edf7cbb5c087a8bf5
Ruizhong Qiu
6833d08edf7cbb5c087a8bf6
Ling Yang
claimed_verified
2025-06-27T08:58:15.759Z
64fde4e252e82dd432b74ce9
https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg
Ling Yang
user
Lingaaaaaaa
6833d08edf7cbb5c087a8bf7
Jingrui He
Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
Large language models are typically adapted to downstream tasks through
supervised fine-tuning on domain-specific data. While standard fine-tuning
focuses on minimizing generation loss to optimize model parameters, we take a
deeper step by retaining and leveraging the model's own learning signals,
analogous to how human learners reflect on past mistakes to improve future
performance. We first introduce the concept of Mistake Log to systematically
track the model's learning behavior and recurring errors throughout
fine-tuning. Treating the original transformer-based model as the Pilot, we
correspondingly design a Copilot model to refine the Pilot's inference
performance via logits rectification. We name the overall Pilot-Copilot
framework the Transformer Copilot, which introduces (i) a novel Copilot model
design, (ii) a joint training paradigm where the Copilot continuously learns
from the evolving Mistake Log alongside the Pilot, and (iii) a fused inference
paradigm where the Copilot rectifies the Pilot's logits for enhanced
generation. We provide both theoretical and empirical analyses on our new
learning framework. Experiments on 12 benchmarks spanning commonsense,
arithmetic, and recommendation tasks demonstrate that Transformer Copilot
consistently improves performance by up to 34.5%, while introducing marginal
computational overhead to Pilot models and exhibiting strong scalability and
transferability.
The Transformer Copilot framework enhances large language model performance through a Copilot model that refines the Pilot's logits based on a Mistake Log, leading to consistent performance improvements across various benchmarks.
large language models supervised fine-tuning domain-specific data generation loss model parameters learning signals Mistake Log transformer-based model Copilot model logits rectification joint training paradigm fused inference paradigm performance improvements computational overhead scalability transferability