title: DMM: Building a Versatile Image Generation Model via Distillation-Based
    Model Merging
summary: The success of text-to-image (T2I) generation models has spurred a
  proliferation of numerous model checkpoints fine-tuned from the same base model
  on various specialized datasets. This overwhelming specialized model production
  introduces new challenges for high parameter redundancy and huge storage cost,
  thereby necessitating the development of effective methods to consolidate and
  unify the capabilities of diverse powerful models into a single one. A common
  practice in model merging adopts static linear interpolation in the parameter
  space to achieve the goal of style mixing. However, it neglects the features of
  T2I generation task that numerous distinct models cover sundry styles which may
  lead to incompatibility and confusion in the merged model. To address this
  issue, we introduce a style-promptable image generation pipeline which can
  accurately generate arbitrary-style images under the control of style vectors.
  Based on this design, we propose the score distillation based model merging
  paradigm (DMM), compressing multiple models into a single versatile T2I model.
  Moreover, we rethink and reformulate the model merging task in the context of
  T2I generation, by presenting new merging goals and evaluation protocols. Our
  experiments demonstrate that DMM can compactly reorganize the knowledge from
  multiple teacher models and achieve controllable arbitrary-style generation.
paper_authors._id: 6801d913a0cf74448f93d5c8
paper_authors.name: Tianhui Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-18T09:38:44.267Z
paper_authors._id: 649e7693a83143427691769c
paper_authors.avatarUrl: /avatars/d04f7b3d417423abaa053375212da21f.svg
paper_authors.fullname: Tianhui Song
paper_authors.type: user
paper_authors.user: sthuihui
paper_authors._id: 6801d913a0cf74448f93d5c9
paper_authors.name: Weixin Feng
paper_authors._id: 6801d913a0cf74448f93d5ca
paper_authors.name: Shuai Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-18T09:38:42.164Z
paper_authors._id: 66615c855fd9d736e670e0a9
paper_authors.avatarUrl: /avatars/0ff3127b513552432a7c651e21d7f283.svg
paper_authors.fullname: wangshuai
paper_authors.type: user
paper_authors.user: wangsssssss
paper_authors._id: 6801d913a0cf74448f93d5cb
paper_authors.name: Xubin Li
paper_authors._id: 6801d913a0cf74448f93d5cc
paper_authors.name: Tiezheng Ge
paper_authors._id: 6801d913a0cf74448f93d5cd
paper_authors.name: Bo Zheng
paper_authors._id: 6801d913a0cf74448f93d5ce
paper_authors.name: Limin Wang
paper_title: DMM: Building a Versatile Image Generation Model via Distillation-Based
    Model Merging
paper_summary: The success of text-to-image (T2I) generation models has spurred a
  proliferation of numerous model checkpoints fine-tuned from the same base model
  on various specialized datasets. This overwhelming specialized model production
  introduces new challenges for high parameter redundancy and huge storage cost,
  thereby necessitating the development of effective methods to consolidate and
  unify the capabilities of diverse powerful models into a single one. A common
  practice in model merging adopts static linear interpolation in the parameter
  space to achieve the goal of style mixing. However, it neglects the features of
  T2I generation task that numerous distinct models cover sundry styles which may
  lead to incompatibility and confusion in the merged model. To address this
  issue, we introduce a style-promptable image generation pipeline which can
  accurately generate arbitrary-style images under the control of style vectors.
  Based on this design, we propose the score distillation based model merging
  paradigm (DMM), compressing multiple models into a single versatile T2I model.
  Moreover, we rethink and reformulate the model merging task in the context of
  T2I generation, by presenting new merging goals and evaluation protocols. Our
  experiments demonstrate that DMM can compactly reorganize the knowledge from
  multiple teacher models and achieve controllable arbitrary-style generation.
paper_ai_summary: The paper introduces score distillation based model merging for text-to-image generation, enabling a single versatile model to produce images in different styles based on style vectors.
paper_ai_keywords: text-to-image (T2I) parameter redundancy storage cost model merging static linear interpolation style vectors score distillation based model merging (DMM) style-promptable image generation