title: Show-o: One Single Transformer to Unify Multimodal Understanding and
    Generation
summary: We present a unified transformer, i.e., Show-o, that unifies multimodal
  understanding and generation. Unlike fully autoregressive models, Show-o
  unifies autoregressive and (discrete) diffusion modeling to adaptively handle
  inputs and outputs of various and mixed modalities. The unified model flexibly
  supports a wide range of vision-language tasks including visual
  question-answering, text-to-image generation, text-guided
  inpainting/extrapolation, and mixed-modality generation. Across various
  benchmarks, it demonstrates comparable or superior performance to existing
  individual models with an equivalent or larger number of parameters tailored
  for understanding or generation. This significantly highlights its potential as
  a next-generation foundation model. Code and models are released at
  https://github.com/showlab/Show-o.
paper_authors._id: 66c7fffaac77b1ccf9f43d04
paper_authors.name: Jinheng Xie
paper_authors._id: 66c7fffaac77b1ccf9f43d05
paper_authors.name: Weijia Mao
paper_authors._id: 66c7fffaac77b1ccf9f43d06
paper_authors.name: Zechen Bai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-01T07:16:56.506Z
paper_authors._id: 64b7833aa5018e3c7c9b50d8
paper_authors.avatarUrl: /avatars/782415605ed786b73f484fcc86a6384f.svg
paper_authors.fullname: Zechen Bai
paper_authors.type: user
paper_authors.user: ZechenBai
paper_authors._id: 66c7fffaac77b1ccf9f43d07
paper_authors.name: David Junhao Zhang
paper_authors._id: 66c7fffaac77b1ccf9f43d08
paper_authors.name: Weihao Wang
paper_authors._id: 66c7fffaac77b1ccf9f43d09
paper_authors.name: Kevin Qinghong Lin
paper_authors._id: 66c7fffaac77b1ccf9f43d0a
paper_authors.name: Yuchao Gu
paper_authors._id: 66c7fffaac77b1ccf9f43d0b
paper_authors.name: Zhijie Chen
paper_authors._id: 66c7fffaac77b1ccf9f43d0c
paper_authors.name: Zhenheng Yang
paper_authors._id: 66c7fffaac77b1ccf9f43d0d
paper_authors.name: Mike Zheng Shou
paper_title: Show-o: One Single Transformer to Unify Multimodal Understanding and
    Generation
paper_summary: We present a unified transformer, i.e., Show-o, that unifies multimodal
  understanding and generation. Unlike fully autoregressive models, Show-o
  unifies autoregressive and (discrete) diffusion modeling to adaptively handle
  inputs and outputs of various and mixed modalities. The unified model flexibly
  supports a wide range of vision-language tasks including visual
  question-answering, text-to-image generation, text-guided
  inpainting/extrapolation, and mixed-modality generation. Across various
  benchmarks, it demonstrates comparable or superior performance to existing
  individual models with an equivalent or larger number of parameters tailored
  for understanding or generation. This significantly highlights its potential as
  a next-generation foundation model. Code and models are released at
  https://github.com/showlab/Show-o.
paper_ai_keywords: unified transformer Show-o multimodal understanding generation fully autoregressive models autoregressive (discrete) diffusion modeling visual question-answering text-to-image generation text-guided inpainting/extrapolation mixed-modality generation vision-language tasks foundation model