title: MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation
summary: In this paper, we present MoMA: an open-vocabulary, training-free
  personalized image model that boasts flexible zero-shot capabilities. As
  foundational text-to-image models rapidly evolve, the demand for robust
  image-to-image translation grows. Addressing this need, MoMA specializes in
  subject-driven personalized image generation. Utilizing an open-source,
  Multimodal Large Language Model (MLLM), we train MoMA to serve a dual role as
  both a feature extractor and a generator. This approach effectively synergizes
  reference image and text prompt information to produce valuable image features,
  facilitating an image diffusion model. To better leverage the generated
  features, we further introduce a novel self-attention shortcut method that
  efficiently transfers image features to an image diffusion model, improving the
  resemblance of the target object in generated images. Remarkably, as a
  tuning-free plug-and-play module, our model requires only a single reference
  image and outperforms existing methods in generating images with high detail
  fidelity, enhanced identity-preservation and prompt faithfulness. Our work is
  open-source, thereby providing universal access to these advancements.
paper_authors._id: 6614c9137de7824887701c20
paper_authors.name: Kunpeng Song
paper_authors._id: 6614c9137de7824887701c21
paper_authors.name: Yizhe Zhu
paper_authors._id: 6614c9137de7824887701c22
paper_authors.name: Bingchen Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:47:50.306Z
paper_authors._id: 62f9e1f90697d2242198c1db
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1660543471146-noauth.jpeg
paper_authors.fullname: bingchen liu
paper_authors.type: user
paper_authors.user: codegeasslbc
paper_authors._id: 6614c9137de7824887701c23
paper_authors.name: Qing Yan
paper_authors._id: 6614c9137de7824887701c24
paper_authors.name: Ahmed Elgammal
paper_authors._id: 6614c9137de7824887701c25
paper_authors.name: Xiao Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-03T07:41:44.144Z
paper_authors._id: 62b3d2490385d6048985b42b
paper_authors.avatarUrl: /avatars/1c4e534d086d53ef1057fbc00334350a.svg
paper_authors.fullname: Xiao Yang
paper_authors.type: user
paper_authors.user: seravee008
paper_title: MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation
paper_summary: In this paper, we present MoMA: an open-vocabulary, training-free
  personalized image model that boasts flexible zero-shot capabilities. As
  foundational text-to-image models rapidly evolve, the demand for robust
  image-to-image translation grows. Addressing this need, MoMA specializes in
  subject-driven personalized image generation. Utilizing an open-source,
  Multimodal Large Language Model (MLLM), we train MoMA to serve a dual role as
  both a feature extractor and a generator. This approach effectively synergizes
  reference image and text prompt information to produce valuable image features,
  facilitating an image diffusion model. To better leverage the generated
  features, we further introduce a novel self-attention shortcut method that
  efficiently transfers image features to an image diffusion model, improving the
  resemblance of the target object in generated images. Remarkably, as a
  tuning-free plug-and-play module, our model requires only a single reference
  image and outperforms existing methods in generating images with high detail
  fidelity, enhanced identity-preservation and prompt faithfulness. Our work is
  open-source, thereby providing universal access to these advancements.
paper_ai_summary: MoMA, an open-vocabulary and training-free model, uses a Multimodal Large Language Model to generate high-detail, identity-preserved images through an image diffusion model enhanced by a self-attention shortcut.
paper_ai_keywords: Multimodal Large Language Model feature extractor image diffusion model self-attention shortcut open-vocabulary zero-shot capabilities image-to-image translation reference image text prompt high detail fidelity identity-preservation prompt faithfulness