title: GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong
    Prompt Optimizers
summary: The effectiveness of large language models (LLMs) is closely tied to the
  design of prompts, making prompt optimization essential for enhancing their
  performance across a wide range of tasks. Many existing approaches to
  automating prompt engineering rely exclusively on textual feedback, refining
  prompts based solely on inference errors identified by large, computationally
  expensive LLMs. Unfortunately, smaller models struggle to generate high-quality
  feedback, resulting in complete dependence on large LLM judgment. Moreover,
  these methods fail to leverage more direct and finer-grained information, such
  as gradients, due to operating purely in text space. To this end, we introduce
  GReaTer, a novel prompt optimization technique that directly incorporates
  gradient information over task-specific reasoning. By utilizing task loss
  gradients, GReaTer enables self-optimization of prompts for open-source,
  lightweight language models without the need for costly closed-source LLMs.
  This allows high-performance prompt optimization without dependence on massive
  LLMs, closing the gap between smaller models and the sophisticated reasoning
  often needed for prompt refinement. Extensive evaluations across diverse
  reasoning tasks including BBH, GSM8k, and FOLIO demonstrate that GReaTer
  consistently outperforms previous state-of-the-art prompt optimization methods,
  even those reliant on powerful LLMs. Additionally, GReaTer-optimized prompts
  frequently exhibit better transferability and, in some cases, boost task
  performance to levels comparable to or surpassing those achieved by larger
  language models, highlighting the effectiveness of prompt optimization guided
  by gradients over reasoning. Code of GReaTer is available at
  https://github.com/psunlpgroup/GreaTer.
paper_authors._id: 67607fea899f4731eb7dc446
paper_authors.name: Sarkar Snigdha Sarathi Das
paper_authors._id: 67607fea899f4731eb7dc447
paper_authors.name: Ryo Kamoi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-17T08:04:07.183Z
paper_authors._id: 60591896102f61b42f65ae37
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60591896102f61b42f65ae37/PSHcLoEeNnwvzAPOd6-6i.jpeg
paper_authors.fullname: Ryo Kamoi
paper_authors.type: user
paper_authors.user: ryokamoi
paper_authors._id: 67607fea899f4731eb7dc448
paper_authors.name: Bo Pang
paper_authors._id: 67607fea899f4731eb7dc449
paper_authors.name: Yusen Zhang
paper_authors._id: 67607fea899f4731eb7dc44a
paper_authors.name: Caiming Xiong
paper_authors._id: 67607fea899f4731eb7dc44b
paper_authors.name: Rui Zhang
paper_title: GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong
    Prompt Optimizers
paper_summary: The effectiveness of large language models (LLMs) is closely tied to the
  design of prompts, making prompt optimization essential for enhancing their
  performance across a wide range of tasks. Many existing approaches to
  automating prompt engineering rely exclusively on textual feedback, refining
  prompts based solely on inference errors identified by large, computationally
  expensive LLMs. Unfortunately, smaller models struggle to generate high-quality
  feedback, resulting in complete dependence on large LLM judgment. Moreover,
  these methods fail to leverage more direct and finer-grained information, such
  as gradients, due to operating purely in text space. To this end, we introduce
  GReaTer, a novel prompt optimization technique that directly incorporates
  gradient information over task-specific reasoning. By utilizing task loss
  gradients, GReaTer enables self-optimization of prompts for open-source,
  lightweight language models without the need for costly closed-source LLMs.
  This allows high-performance prompt optimization without dependence on massive
  LLMs, closing the gap between smaller models and the sophisticated reasoning
  often needed for prompt refinement. Extensive evaluations across diverse
  reasoning tasks including BBH, GSM8k, and FOLIO demonstrate that GReaTer
  consistently outperforms previous state-of-the-art prompt optimization methods,
  even those reliant on powerful LLMs. Additionally, GReaTer-optimized prompts
  frequently exhibit better transferability and, in some cases, boost task
  performance to levels comparable to or surpassing those achieved by larger
  language models, highlighting the effectiveness of prompt optimization guided
  by gradients over reasoning. Code of GReaTer is available at
  https://github.com/psunlpgroup/GreaTer.
paper_ai_keywords: gradient information task loss gradients prompt optimization GReaTer BBH GSM8k FOLIO transferability