title: IHEval: Evaluating Language Models on Following the Instruction
    Hierarchy
summary: The instruction hierarchy, which establishes a priority order from system
  messages to user messages, conversation history, and tool outputs, is essential
  for ensuring consistent and safe behavior in language models (LMs). Despite its
  importance, this topic receives limited attention, and there is a lack of
  comprehensive benchmarks for evaluating models' ability to follow the
  instruction hierarchy. We bridge this gap by introducing IHEval, a novel
  benchmark comprising 3,538 examples across nine tasks, covering cases where
  instructions in different priorities either align or conflict. Our evaluation
  of popular LMs highlights their struggle to recognize instruction priorities.
  All evaluated models experience a sharp performance decline when facing
  conflicting instructions, compared to their original instruction-following
  performance. Moreover, the most competitive open-source model only achieves 48%
  accuracy in resolving such conflicts. Our results underscore the need for
  targeted optimization in the future development of LMs.
paper_authors._id: 67b4cf1994ec5e365fb7995d
paper_authors.name: Zhihan Zhang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-18T18:19:31.455Z
paper_authors._id: 63bf9695da08ed054400205e
paper_authors.avatarUrl: /avatars/b6fca49559a61cf66628088c60d26c10.svg
paper_authors.fullname: Zhihan Zhang
paper_authors.type: user
paper_authors.user: zhihz0535
paper_authors._id: 67b4cf1994ec5e365fb7995e
paper_authors.name: Shiyang Li
paper_authors._id: 67b4cf1994ec5e365fb7995f
paper_authors.name: Zixuan Zhang
paper_authors._id: 67b4cf1994ec5e365fb79960
paper_authors.name: Xin Liu
paper_authors._id: 67b4cf1994ec5e365fb79961
paper_authors.name: Haoming Jiang
paper_authors._id: 67b4cf1994ec5e365fb79962
paper_authors.name: Xianfeng Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T15:25:56.312Z
paper_authors._id: 6465f6467ff8fcbef7d22513
paper_authors.avatarUrl: /avatars/07992835c235fbb07016a0ea4f1d61cb.svg
paper_authors.fullname: Xianfeng Tang
paper_authors.type: user
paper_authors.user: xianft
paper_authors._id: 67b4cf1994ec5e365fb79963
paper_authors.name: Yifan Gao
paper_authors._id: 67b4cf1994ec5e365fb79964
paper_authors.name: Zheng Li
paper_authors._id: 67b4cf1994ec5e365fb79965
paper_authors.name: Haodong Wang
paper_authors._id: 67b4cf1994ec5e365fb79966
paper_authors.name: Zhaoxuan Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T15:26:12.966Z
paper_authors._id: 638eaca4a26f3d5af37be8b3
paper_authors.avatarUrl: /avatars/476890d83191d0cbdb9a3d5351a129da.svg
paper_authors.fullname: Zhaoxuan_Tan
paper_authors.type: user
paper_authors.user: Zhaoxuan
paper_authors._id: 67b4cf1994ec5e365fb79967
paper_authors.name: Yichuan Li
paper_authors._id: 67b4cf1994ec5e365fb79968
paper_authors.name: Qingyu Yin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T15:26:54.938Z
paper_authors._id: 6453cb22908e259483c0a061
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6453cb22908e259483c0a061/hMgdwZUsUbgquGalzPGzV.jpeg
paper_authors.fullname: Qingyu_Yin
paper_authors.type: user
paper_authors.user: MikaStars39
paper_authors._id: 67b4cf1994ec5e365fb79969
paper_authors.name: Bing Yin
paper_authors._id: 67b4cf1994ec5e365fb7996a
paper_authors.name: Meng Jiang
paper_title: IHEval: Evaluating Language Models on Following the Instruction
    Hierarchy
paper_summary: The instruction hierarchy, which establishes a priority order from system
  messages to user messages, conversation history, and tool outputs, is essential
  for ensuring consistent and safe behavior in language models (LMs). Despite its
  importance, this topic receives limited attention, and there is a lack of
  comprehensive benchmarks for evaluating models' ability to follow the
  instruction hierarchy. We bridge this gap by introducing IHEval, a novel
  benchmark comprising 3,538 examples across nine tasks, covering cases where
  instructions in different priorities either align or conflict. Our evaluation
  of popular LMs highlights their struggle to recognize instruction priorities.
  All evaluated models experience a sharp performance decline when facing
  conflicting instructions, compared to their original instruction-following
  performance. Moreover, the most competitive open-source model only achieves 48%
  accuracy in resolving such conflicts. Our results underscore the need for
  targeted optimization in the future development of LMs.
paper_ai_keywords: instruction hierarchy language models (LMs) benchmark instruction-following conflicting instructions open-source model