CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control & Altering of T2I Models
<|reference_start|>CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control & Altering of T2I Models: Text-to-image generative models have become a prominent and powerful tool that excels at generating high-resolution realistic images. However, guiding the generative process of these models to consider detailed forms of conditioning reflecting style and/or structure information remains an open problem. In this paper, we present LoRAdapter, an approach that unifies both style and structure conditioning under the same formulation using a novel conditional LoRA block that enables zero-shot control. LoRAdapter is an efficient, powerful, and architecture-agnostic approach to condition text-to-image diffusion models, which enables fine-grained control conditioning during generation and outperforms recent state-of-the-art approaches.<|reference_end|>
@article{stracke2024ctrloralter:,
    title={CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control &
  Altering of T2I Models},
    author={Nick Stracke, Stefan Andreas Baumann, Joshua M. Susskind, Miguel Angel
  Bautista, Bj"orn Ommer},
    journal={arXiv preprint arXiv:2405.07913},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.07913},
    primaryClass={cs.CV}
}