title: Adaptive Layer-skipping in Pre-trained LLMs
summary: Various layer-skipping methods have been proposed to accelerate token
  generation in large language models (LLMs). However, they have overlooked a
  fundamental question: How do computational demands vary across the generation
  of different tokens? In this work, we introduce FlexiDepth, a method that
  dynamically adjusts the number of Transformer layers used in text generation.
  By incorporating a plug-in router and adapter, FlexiDepth enables adaptive
  layer-skipping in LLMs without modifying their original parameters. Introducing
  FlexiDepth to Llama-3-8B model achieves layer skipping of 8 layers out of 32,
  and meanwhile maintains the full 100\% benchmark performance. Experimental
  results with FlexiDepth demonstrate that computational demands in LLMs
  significantly vary based on token type. Specifically, generating repetitive
  tokens or fixed phrases requires fewer layers, whereas producing tokens
  involving computation or high uncertainty requires more layers. Interestingly,
  this adaptive allocation pattern aligns with human intuition. To advance
  research in this area, we open sourced FlexiDepth and a dataset documenting
  FlexiDepth's layer allocation patterns for future exploration.
paper_authors._id: 67ec9ed83d267d266640c03d
paper_authors.name: Xuan Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:44.959Z
paper_authors._id: 659d98feec9f8f337f145ec4
paper_authors.avatarUrl: /avatars/7de3bea0f203326e78916882655b5d55.svg
paper_authors.fullname: xuan luo
paper_authors.type: user
paper_authors.user: xuan-luo
paper_authors._id: 67ec9ed83d267d266640c03e
paper_authors.name: Weizhi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:42.010Z
paper_authors._id: 63d34004b734eaa4d4faeccf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63d34004b734eaa4d4faeccf/aH7F-xn4PMQjHHcvL-0vs.jpeg
paper_authors.fullname: Weizhi Wang
paper_authors.type: user
paper_authors.user: weizhiwang
paper_authors._id: 67ec9ed83d267d266640c03f
paper_authors.name: Xifeng Yan
paper_title: Adaptive Layer-skipping in Pre-trained LLMs
paper_summary: Various layer-skipping methods have been proposed to accelerate token
  generation in large language models (LLMs). However, they have overlooked a
  fundamental question: How do computational demands vary across the generation
  of different tokens? In this work, we introduce FlexiDepth, a method that
  dynamically adjusts the number of Transformer layers used in text generation.
  By incorporating a plug-in router and adapter, FlexiDepth enables adaptive
  layer-skipping in LLMs without modifying their original parameters. Introducing
  FlexiDepth to Llama-3-8B model achieves layer skipping of 8 layers out of 32,
  and meanwhile maintains the full 100\% benchmark performance. Experimental
  results with FlexiDepth demonstrate that computational demands in LLMs
  significantly vary based on token type. Specifically, generating repetitive
  tokens or fixed phrases requires fewer layers, whereas producing tokens
  involving computation or high uncertainty requires more layers. Interestingly,
  this adaptive allocation pattern aligns with human intuition. To advance
  research in this area, we open sourced FlexiDepth and a dataset documenting
  FlexiDepth's layer allocation patterns for future exploration.
paper_ai_summary: FlexiDepth dynamically adjusts Transformer layers in LLMs based on token complexity, enhancing computational efficiency without sacrificing performance.
paper_ai_keywords: layer-skipping token generation large language models LLMs Transformer layers FlexiDepth plug-in router adapter layer allocation benchmark performance