TraDiffusion: Trajectory-Based Training-Free Image Generation
In this work, we propose a training-free, trajectory-based controllable T2I
approach, termed TraDiffusion. This novel method allows users to effortlessly
guide image generation via mouse trajectories. To achieve precise control, we
design a distance awareness energy function to effectively guide latent
variables, ensuring that the focus of generation is within the areas defined by
the trajectory. The energy function encompasses a control function to draw the
generation closer to the specified trajectory and a movement function to
diminish activity in areas distant from the trajectory. Through extensive
experiments and qualitative assessments on the COCO dataset, the results reveal
that TraDiffusion facilitates simpler, more natural image control. Moreover, it
showcases the ability to manipulate salient regions, attributes, and
relationships within the generated images, alongside visual input based on
arbitrary or enhanced trajectories.
66c41a52b237028a181ef5be
Mingrui Wu
66c41a52b237028a181ef5bf
Oucheng Huang
66c41a52b237028a181ef5c0
Jiayi Ji
66c41a52b237028a181ef5c1
Jiale Li
66c41a52b237028a181ef5c2
Xinyue Cai
66c41a52b237028a181ef5c3
Huafeng Kuang
66c41a52b237028a181ef5c4
Jianzhuang Liu
66c41a52b237028a181ef5c5
Xiaoshuai Sun
66c41a52b237028a181ef5c6
Rongrong Ji
TraDiffusion: Trajectory-Based Training-Free Image Generation
In this work, we propose a training-free, trajectory-based controllable T2I
approach, termed TraDiffusion. This novel method allows users to effortlessly
guide image generation via mouse trajectories. To achieve precise control, we
design a distance awareness energy function to effectively guide latent
variables, ensuring that the focus of generation is within the areas defined by
the trajectory. The energy function encompasses a control function to draw the
generation closer to the specified trajectory and a movement function to
diminish activity in areas distant from the trajectory. Through extensive
experiments and qualitative assessments on the COCO dataset, the results reveal
that TraDiffusion facilitates simpler, more natural image control. Moreover, it
showcases the ability to manipulate salient regions, attributes, and
relationships within the generated images, alongside visual input based on
arbitrary or enhanced trajectories.
TraDiffusion, a training-free trajectory-based approach, allows precise control over image generation using mouse trajectories by adjusting latent variables with a custom energy function.
TraDiffusion training-free trajectory-based controllable T2I image generation energy function latent variables control function movement function COCO dataset salient regions attributes relationships visual input