title: An Image is Worth Multiple Words: Learning Object Level Concepts using
    Multi-Concept Prompt Learning
summary: Textural Inversion, a prompt learning method, learns a singular embedding for
  a new "word" to represent image style and appearance, allowing it to be
  integrated into natural language sentences to generate novel synthesised
  images. However, identifying and integrating multiple object-level concepts
  within one scene poses significant challenges even when embeddings for
  individual concepts are attainable. This is further confirmed by our empirical
  tests. To address this challenge, we introduce a framework for Multi-Concept
  Prompt Learning (MCPL), where multiple new "words" are simultaneously learned
  from a single sentence-image pair. To enhance the accuracy of word-concept
  correlation, we propose three regularisation techniques: Attention Masking
  (AttnMask) to concentrate learning on relevant areas; Prompts Contrastive Loss
  (PromptCL) to separate the embeddings of different concepts; and Bind adjective
  (Bind adj.) to associate new "words" with known words. We evaluate via image
  generation, editing, and attention visualisation with diverse images. Extensive
  quantitative comparisons demonstrate that our method can learn more
  semantically disentangled concepts with enhanced word-concept correlation.
  Additionally, we introduce a novel dataset and evaluation protocol tailored for
  this new task of learning object-level concepts.
paper_authors._id: 6532117b829e1dc2f291438b
paper_authors.name: Chen Jin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-20T12:44:32.606Z
paper_authors._id: 62d691726279202d95f86bb6
paper_authors.avatarUrl: /avatars/a52567e73e6b4e84219cb32c85ee8c1f.svg
paper_authors.fullname: Chen Jin
paper_authors.type: user
paper_authors.user: lxasqjc
paper_authors._id: 6532117b829e1dc2f291438c
paper_authors.name: Ryutaro Tanno
paper_authors._id: 6532117b829e1dc2f291438d
paper_authors.name: Amrutha Saseendran
paper_authors._id: 6532117b829e1dc2f291438e
paper_authors.name: Tom Diethe
paper_authors._id: 6532117b829e1dc2f291438f
paper_authors.name: Philip Teare
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-21T09:22:28.981Z
paper_authors._id: 631da8c0f318ed8dfd32370f
paper_authors.avatarUrl: /avatars/d012e65e83fee403a510dff3f032232d.svg
paper_authors.fullname: Philip Teare
paper_authors.type: user
paper_authors.user: evolu8
paper_title: An Image is Worth Multiple Words: Learning Object Level Concepts using
    Multi-Concept Prompt Learning
paper_summary: Textural Inversion, a prompt learning method, learns a singular embedding for
  a new "word" to represent image style and appearance, allowing it to be
  integrated into natural language sentences to generate novel synthesised
  images. However, identifying and integrating multiple object-level concepts
  within one scene poses significant challenges even when embeddings for
  individual concepts are attainable. This is further confirmed by our empirical
  tests. To address this challenge, we introduce a framework for Multi-Concept
  Prompt Learning (MCPL), where multiple new "words" are simultaneously learned
  from a single sentence-image pair. To enhance the accuracy of word-concept
  correlation, we propose three regularisation techniques: Attention Masking
  (AttnMask) to concentrate learning on relevant areas; Prompts Contrastive Loss
  (PromptCL) to separate the embeddings of different concepts; and Bind adjective
  (Bind adj.) to associate new "words" with known words. We evaluate via image
  generation, editing, and attention visualisation with diverse images. Extensive
  quantitative comparisons demonstrate that our method can learn more
  semantically disentangled concepts with enhanced word-concept correlation.
  Additionally, we introduce a novel dataset and evaluation protocol tailored for
  this new task of learning object-level concepts.
paper_ai_summary: A novel Multi-Concept Prompt Learning framework (MCPL) learns multiple embeddings from a single sentence-image pair to represent object-level concepts in images, using regularisation techniques to enhance accuracy.
paper_ai_keywords: prompt learning Textural Inversion image style image appearance Multi-Concept Prompt Learning MCPL Attention Masking AttnMask Prompts Contrastive Loss PromptCL Bind adjective Bind adj. word-concept correlation image generation image editing attention visualisation semantic disentangled concepts dataset evaluation protocol