title: GoT: Unleashing Reasoning Capability of Multimodal Large Language Model
    for Visual Generation and Editing
summary: Current image generation and editing methods primarily process textual
  prompts as direct inputs without reasoning about visual composition and
  explicit operations. We present Generation Chain-of-Thought (GoT), a novel
  paradigm that enables generation and editing through an explicit language
  reasoning process before outputting images. This approach transforms
  conventional text-to-image generation and editing into a reasoning-guided
  framework that analyzes semantic relationships and spatial arrangements. We
  define the formulation of GoT and construct large-scale GoT datasets containing
  over 9M samples with detailed reasoning chains capturing semantic-spatial
  relationships. To leverage the advantages of GoT, we implement a unified
  framework that integrates Qwen2.5-VL for reasoning chain generation with an
  end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance
  Module. Experiments show our GoT framework achieves excellent performance on
  both generation and editing tasks, with significant improvements over
  baselines. Additionally, our approach enables interactive visual generation,
  allowing users to explicitly modify reasoning steps for precise image
  adjustments. GoT pioneers a new direction for reasoning-driven visual
  generation and editing, producing images that better align with human intent.
  To facilitate future research, we make our datasets, code, and pretrained
  models publicly available at https://github.com/rongyaofang/GoT.
paper_authors._id: 67d3a632db36a4d5d95dbcff
paper_authors.name: Rongyao Fang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:56:14.110Z
paper_authors._id: 65b8724123d948d884b379b1
paper_authors.avatarUrl: /avatars/ce189d1d8d688c17912f9b869035b2d0.svg
paper_authors.fullname: Rongyao Fang
paper_authors.type: user
paper_authors.user: LucasFang
paper_authors._id: 67d3a632db36a4d5d95dbd00
paper_authors.name: Chengqi Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:07:01.612Z
paper_authors._id: 64a2b496e2e19de17db7de65
paper_authors.avatarUrl: /avatars/241448ca487833d6cc5d57bb1fdb6ee5.svg
paper_authors.fullname: Duan Chengqi
paper_authors.type: user
paper_authors.user: gogoduan
paper_authors._id: 67d3a632db36a4d5d95dbd01
paper_authors.name: Kun Wang
paper_authors._id: 67d3a632db36a4d5d95dbd02
paper_authors.name: Linjiang Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:07:08.707Z
paper_authors._id: 65fc7c824d36be78e66ba92d
paper_authors.avatarUrl: /avatars/d4a55c820cae533f91724e062427516a.svg
paper_authors.fullname: Linjiang Huang
paper_authors.type: user
paper_authors.user: LjHuang
paper_authors._id: 67d3a632db36a4d5d95dbd03
paper_authors.name: Hao Li
paper_authors._id: 67d3a632db36a4d5d95dbd04
paper_authors.name: Shilin Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:07:18.212Z
paper_authors._id: 65273fea0ef49cfb783fa5c1
paper_authors.avatarUrl: /avatars/0c9e204bc2151c8cc533311900d05a36.svg
paper_authors.fullname: shilinyan
paper_authors.type: user
paper_authors.user: shilinyan
paper_authors._id: 67d3a632db36a4d5d95dbd05
paper_authors.name: Hao Tian
paper_authors._id: 67d3a632db36a4d5d95dbd06
paper_authors.name: Xingyu Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:07:42.264Z
paper_authors._id: 666d4a0fe70e5838d95aebee
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6dkjoFA_sOjCkjvcvozZ5.jpeg
paper_authors.fullname: zengxingyu
paper_authors.type: user
paper_authors.user: zengxingyu
paper_authors._id: 67d3a632db36a4d5d95dbd07
paper_authors.name: Rui Zhao
paper_authors._id: 67d3a632db36a4d5d95dbd08
paper_authors.name: Jifeng Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:14:59.460Z
paper_authors._id: 64686f7172d9180d4ac8b4e4
paper_authors.avatarUrl: /avatars/db67dd6c4b2b41054ddcce5a18ade6f8.svg
paper_authors.fullname: Jifeng Dai
paper_authors.type: user
paper_authors.user: daijifeng
paper_authors._id: 67d3a632db36a4d5d95dbd09
paper_authors.name: Xihui Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:14:50.924Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_authors._id: 67d3a632db36a4d5d95dbd0a
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T09:14:43.402Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_title: GoT: Unleashing Reasoning Capability of Multimodal Large Language Model
    for Visual Generation and Editing
paper_summary: Current image generation and editing methods primarily process textual
  prompts as direct inputs without reasoning about visual composition and
  explicit operations. We present Generation Chain-of-Thought (GoT), a novel
  paradigm that enables generation and editing through an explicit language
  reasoning process before outputting images. This approach transforms
  conventional text-to-image generation and editing into a reasoning-guided
  framework that analyzes semantic relationships and spatial arrangements. We
  define the formulation of GoT and construct large-scale GoT datasets containing
  over 9M samples with detailed reasoning chains capturing semantic-spatial
  relationships. To leverage the advantages of GoT, we implement a unified
  framework that integrates Qwen2.5-VL for reasoning chain generation with an
  end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance
  Module. Experiments show our GoT framework achieves excellent performance on
  both generation and editing tasks, with significant improvements over
  baselines. Additionally, our approach enables interactive visual generation,
  allowing users to explicitly modify reasoning steps for precise image
  adjustments. GoT pioneers a new direction for reasoning-driven visual
  generation and editing, producing images that better align with human intent.
  To facilitate future research, we make our datasets, code, and pretrained
  models publicly available at https://github.com/rongyaofang/GoT.
paper_ai_summary: A new reasoning-driven paradigm, GoT, improves text-to-image generation and editing through language understanding and reasoning chains, integrating Qwen2.5-VL and an enhanced diffusion model with a Semantic-Spatial Guidance Module.
paper_ai_keywords: Generation Chain-of-Thought GoT diffusion model Semantic-Spatial Guidance Module Qwen2.5-VL reasoning chain text-to-image generation interactive visual generation