title: IFAdapter: Instance Feature Control for Grounded Text-to-Image
    Generation
summary: While Text-to-Image (T2I) diffusion models excel at generating visually
  appealing images of individual instances, they struggle to accurately position
  and control the features generation of multiple instances. The Layout-to-Image
  (L2I) task was introduced to address the positioning challenges by
  incorporating bounding boxes as spatial control signals, but it still falls
  short in generating precise instance features. In response, we propose the
  Instance Feature Generation (IFG) task, which aims to ensure both positional
  accuracy and feature fidelity in generated instances. To address the IFG task,
  we introduce the Instance Feature Adapter (IFAdapter). The IFAdapter enhances
  feature depiction by incorporating additional appearance tokens and utilizing
  an Instance Semantic Map to align instance-level features with spatial
  locations. The IFAdapter guides the diffusion process as a plug-and-play
  module, making it adaptable to various community models. For evaluation, we
  contribute an IFG benchmark and develop a verification pipeline to objectively
  compare models' abilities to generate instances with accurate positioning and
  features. Experimental results demonstrate that IFAdapter outperforms other
  models in both quantitative and qualitative evaluations.
paper_authors._id: 66e3d6c2d3fa06c30282f5bd
paper_authors.name: Yinwei Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-13T07:20:43.493Z
paper_authors._id: 650e77418d01590937b71f1d
paper_authors.avatarUrl: /avatars/9e2ea9c034c39fa6de59789e90919a21.svg
paper_authors.fullname: AWEI
paper_authors.type: user
paper_authors.user: WuYW
paper_authors._id: 66e3d6c2d3fa06c30282f5be
paper_authors.name: Xianpan Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:08:04.274Z
paper_authors._id: 649d2d6a4e03557284229376
paper_authors.avatarUrl: /avatars/6b80dd6291e5d8fd89b10a574bc37558.svg
paper_authors.fullname: XianpanZhou
paper_authors.type: user
paper_authors.user: tinytigerpan
paper_authors._id: 66e3d6c2d3fa06c30282f5bf
paper_authors.name: Bing Ma
paper_authors._id: 66e3d6c2d3fa06c30282f5c0
paper_authors.name: Xuefeng Su
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:08:35.328Z
paper_authors._id: 64abcf623c974392fbc5ebf6
paper_authors.avatarUrl: /avatars/b8ea4ba990beefaebd42b3ff8807d458.svg
paper_authors.fullname: xuefeng
paper_authors.type: user
paper_authors.user: suxuefeng
paper_authors._id: 66e3d6c2d3fa06c30282f5c1
paper_authors.name: Kai Ma
paper_authors._id: 66e3d6c2d3fa06c30282f5c2
paper_authors.name: Xinchao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-13T08:09:07.498Z
paper_authors._id: 63fc03a50aab060792ffef39
paper_authors.avatarUrl: /avatars/9d5b1bb2a41928e08176b703935133ab.svg
paper_authors.fullname: Wangxinchao
paper_authors.type: user
paper_authors.user: wxcTest
paper_title: IFAdapter: Instance Feature Control for Grounded Text-to-Image
    Generation
paper_summary: While Text-to-Image (T2I) diffusion models excel at generating visually
  appealing images of individual instances, they struggle to accurately position
  and control the features generation of multiple instances. The Layout-to-Image
  (L2I) task was introduced to address the positioning challenges by
  incorporating bounding boxes as spatial control signals, but it still falls
  short in generating precise instance features. In response, we propose the
  Instance Feature Generation (IFG) task, which aims to ensure both positional
  accuracy and feature fidelity in generated instances. To address the IFG task,
  we introduce the Instance Feature Adapter (IFAdapter). The IFAdapter enhances
  feature depiction by incorporating additional appearance tokens and utilizing
  an Instance Semantic Map to align instance-level features with spatial
  locations. The IFAdapter guides the diffusion process as a plug-and-play
  module, making it adaptable to various community models. For evaluation, we
  contribute an IFG benchmark and develop a verification pipeline to objectively
  compare models' abilities to generate instances with accurate positioning and
  features. Experimental results demonstrate that IFAdapter outperforms other
  models in both quantitative and qualitative evaluations.
paper_ai_keywords: diffusion models Text-to-Image (T2I) Layout-to-Image (L2I) bounding boxes Instance Feature Generation (IFG) Instance Feature Adapter (IFAdapter) appearance tokens Instance Semantic Map diffusion process benchmark verification pipeline