title: IOPO: Empowering LLMs with Complex Instruction Following via
    Input-Output Preference Optimization
summary: In the realm of large language models (LLMs), the ability of models to
  accurately follow instructions is paramount as more agents and applications
  leverage LLMs for construction, where the complexity of instructions are
  rapidly increasing. However, on the one hand, there is only a certain amount of
  complex instruction evaluation data; on the other hand, there are no dedicated
  algorithms to improve the ability to follow complex instructions. To this end,
  this paper introduces TRACE, a benchmark for improving and evaluating the
  complex instructionfollowing ability, which consists of 120K training data and
  1K evaluation data. Furthermore, we propose IOPO (Input-Output Preference
  Optimization) alignment method which takes both input and output preference
  pairs into consideration, where LLMs not only rapidly align with response
  preferences but also meticulously explore the instruction preferences.
  Extensive experiments on both in-domain and outof-domain datasets confirm the
  effectiveness of IOPO, showing 8.15%, 2.18% improvements on in-domain data and
  6.29%, 3.13% on outof-domain data compared to SFT and DPO respectively.
paper_authors._id: 6732d357faded36d0ebfb787
paper_authors.name: Xinghua Zhang
paper_authors._id: 6732d357faded36d0ebfb788
paper_authors.name: Haiyang Yu
paper_authors._id: 6732d357faded36d0ebfb789
paper_authors.name: Cheng Fu
paper_authors._id: 6732d357faded36d0ebfb78a
paper_authors.name: Fei Huang
paper_authors._id: 6732d357faded36d0ebfb78b
paper_authors.name: Yongbin Li
paper_title: IOPO: Empowering LLMs with Complex Instruction Following via
    Input-Output Preference Optimization
paper_summary: In the realm of large language models (LLMs), the ability of models to
  accurately follow instructions is paramount as more agents and applications
  leverage LLMs for construction, where the complexity of instructions are
  rapidly increasing. However, on the one hand, there is only a certain amount of
  complex instruction evaluation data; on the other hand, there are no dedicated
  algorithms to improve the ability to follow complex instructions. To this end,
  this paper introduces TRACE, a benchmark for improving and evaluating the
  complex instructionfollowing ability, which consists of 120K training data and
  1K evaluation data. Furthermore, we propose IOPO (Input-Output Preference
  Optimization) alignment method which takes both input and output preference
  pairs into consideration, where LLMs not only rapidly align with response
  preferences but also meticulously explore the instruction preferences.
  Extensive experiments on both in-domain and outof-domain datasets confirm the
  effectiveness of IOPO, showing 8.15%, 2.18% improvements on in-domain data and
  6.29%, 3.13% on outof-domain data compared to SFT and DPO respectively.
paper_ai_summary: TRACE is a benchmark for improving complex instruction-following in LLMs, using IOPO to enhance input-output alignment, demonstrating superior performance over SFT and DPO.
paper_ai_keywords: large language models (LLMs) TRACE benchmark complex instruction-following IOPO (Input-Output Preference Optimization) in-domain out-of-domain SFT DPO