title: MakeAnything: Harnessing Diffusion Transformers for Multi-Domain
    Procedural Sequence Generation
summary: A hallmark of human intelligence is the ability to create complex artifacts
  through structured multi-step processes. Generating procedural tutorials with
  AI is a longstanding but challenging goal, facing three key obstacles: (1)
  scarcity of multi-task procedural datasets, (2) maintaining logical continuity
  and visual consistency between steps, and (3) generalizing across multiple
  domains. To address these challenges, we propose a multi-domain dataset
  covering 21 tasks with over 24,000 procedural sequences. Building upon this
  foundation, we introduce MakeAnything, a framework based on the diffusion
  transformer (DIT), which leverages fine-tuning to activate the in-context
  capabilities of DIT for generating consistent procedural sequences. We
  introduce asymmetric low-rank adaptation (LoRA) for image generation, which
  balances generalization capabilities and task-specific performance by freezing
  encoder parameters while adaptively tuning decoder layers. Additionally, our
  ReCraft model enables image-to-process generation through spatiotemporal
  consistency constraints, allowing static images to be decomposed into plausible
  creation sequences. Extensive experiments demonstrate that MakeAnything
  surpasses existing methods, setting new performance benchmarks for procedural
  generation tasks.
paper_authors._id: 67a1dc124fc394b2aa6338d5
paper_authors.name: Yiren Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-04T09:38:41.538Z
paper_authors._id: 64311a95034ecbefddd141ef
paper_authors.avatarUrl: /avatars/b6dc5ca373bedbaa368208517954c375.svg
paper_authors.fullname: Yiren Song
paper_authors.type: user
paper_authors.user: yiren98
paper_authors._id: 67a1dc124fc394b2aa6338d6
paper_authors.name: Cheng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T09:58:24.249Z
paper_authors._id: 6534924de778506c5b1c614c
paper_authors.avatarUrl: /avatars/55417e6e8a561df06836a4ad0912080e.svg
paper_authors.fullname: Cheng Liu
paper_authors.type: user
paper_authors.user: lc03lc
paper_authors._id: 67a1dc124fc394b2aa6338d7
paper_authors.name: Mike Zheng Shou
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-02-04T09:21:27.413Z
paper_authors._id: 63a55320ce5763e06f78519c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1671779060549-noauth.jpeg
paper_authors.fullname: Mike Shou
paper_authors.type: user
paper_authors.user: mikeshou
paper_title: MakeAnything: Harnessing Diffusion Transformers for Multi-Domain
    Procedural Sequence Generation
paper_summary: A hallmark of human intelligence is the ability to create complex artifacts
  through structured multi-step processes. Generating procedural tutorials with
  AI is a longstanding but challenging goal, facing three key obstacles: (1)
  scarcity of multi-task procedural datasets, (2) maintaining logical continuity
  and visual consistency between steps, and (3) generalizing across multiple
  domains. To address these challenges, we propose a multi-domain dataset
  covering 21 tasks with over 24,000 procedural sequences. Building upon this
  foundation, we introduce MakeAnything, a framework based on the diffusion
  transformer (DIT), which leverages fine-tuning to activate the in-context
  capabilities of DIT for generating consistent procedural sequences. We
  introduce asymmetric low-rank adaptation (LoRA) for image generation, which
  balances generalization capabilities and task-specific performance by freezing
  encoder parameters while adaptively tuning decoder layers. Additionally, our
  ReCraft model enables image-to-process generation through spatiotemporal
  consistency constraints, allowing static images to be decomposed into plausible
  creation sequences. Extensive experiments demonstrate that MakeAnything
  surpasses existing methods, setting new performance benchmarks for procedural
  generation tasks.
paper_ai_keywords: diffusion transformer (DIT) asymmetric low-rank adaptation (LoRA) spatiotemporal consistency constraints