title: System Prompt Optimization with Meta-Learning
summary: Large Language Models (LLMs) have shown remarkable capabilities, with
  optimizing their input prompts playing a pivotal role in maximizing their
  performance. However, while LLM prompts consist of both the task-agnostic
  system prompts and task-specific user prompts, existing work on prompt
  optimization has focused on user prompts specific to individual queries or
  tasks, and largely overlooked the system prompt that is, once optimized,
  applicable across different tasks and domains. Motivated by this, we introduce
  the novel problem of bilevel system prompt optimization, whose objective is to
  design system prompts that are robust to diverse user prompts and transferable
  to unseen tasks. To tackle this problem, we then propose a meta-learning
  framework, which meta-learns the system prompt by optimizing it over various
  user prompts across multiple datasets, while simultaneously updating the user
  prompts in an iterative manner to ensure synergy between them. We conduct
  experiments on 14 unseen datasets spanning 5 different domains, on which we
  show that our approach produces system prompts that generalize effectively to
  diverse user prompts. Also, our findings reveal that the optimized system
  prompt enables rapid adaptation even to unseen tasks, requiring fewer
  optimization steps for test-time user prompts while achieving improved
  performance.
paper_authors._id: 68269a1eaa8aded616d280a0
paper_authors.name: Yumin Choi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-16T07:12:06.309Z
paper_authors._id: 64cfa0b9749587dbe01d0079
paper_authors.avatarUrl: /avatars/93ca0a1d9c5578d052c5af0d4d1a0252.svg
paper_authors.fullname: Yumin Choi
paper_authors.type: user
paper_authors.user: YuminChoi
paper_authors._id: 68269a1eaa8aded616d280a1
paper_authors.name: Jinheon Baek
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:10:10.549Z
paper_authors._id: 63036b6c5c70c21d0ea79d48
paper_authors.avatarUrl: /avatars/a7eb03f5cbd4eaa09fe807bbed8bc0f7.svg
paper_authors.fullname: Jinheon Baek
paper_authors.type: user
paper_authors.user: jinheon
paper_authors._id: 68269a1eaa8aded616d280a2
paper_authors.name: Sung Ju Hwang
paper_title: System Prompt Optimization with Meta-Learning
paper_summary: Large Language Models (LLMs) have shown remarkable capabilities, with
  optimizing their input prompts playing a pivotal role in maximizing their
  performance. However, while LLM prompts consist of both the task-agnostic
  system prompts and task-specific user prompts, existing work on prompt
  optimization has focused on user prompts specific to individual queries or
  tasks, and largely overlooked the system prompt that is, once optimized,
  applicable across different tasks and domains. Motivated by this, we introduce
  the novel problem of bilevel system prompt optimization, whose objective is to
  design system prompts that are robust to diverse user prompts and transferable
  to unseen tasks. To tackle this problem, we then propose a meta-learning
  framework, which meta-learns the system prompt by optimizing it over various
  user prompts across multiple datasets, while simultaneously updating the user
  prompts in an iterative manner to ensure synergy between them. We conduct
  experiments on 14 unseen datasets spanning 5 different domains, on which we
  show that our approach produces system prompts that generalize effectively to
  diverse user prompts. Also, our findings reveal that the optimized system
  prompt enables rapid adaptation even to unseen tasks, requiring fewer
  optimization steps for test-time user prompts while achieving improved
  performance.
paper_ai_summary: A meta-learning framework for optimizing system prompts in Large Language Models (LLMs) improves generalization across diverse tasks and datasets.
paper_ai_keywords: Large Language Models (LLMs) bilevel system prompt optimization meta-learning system prompts user prompts unseen datasets unseen tasks rapid adaptation