$\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions
<|reference_start|>$\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions: Synthesizing high-resolution images from intricate, domain-specific information remains a significant challenge in generative modeling, particularly for applications in large-image domains such as digital histopathology and remote sensing. Existing methods face critical limitations: conditional diffusion models in pixel or latent space cannot exceed the resolution on which they were trained without losing fidelity, and computational demands increase significantly for larger image sizes. Patch-based methods offer computational efficiency but fail to capture long-range spatial relationships due to their overreliance on local information. In this paper, we introduce a novel conditional diffusion model in infinite dimensions, $\infty$-Brush for controllable large image synthesis. We propose a cross-attention neural operator to enable conditioning in function space. Our model overcomes the constraints of traditional finite-dimensional diffusion models and patch-based methods, offering scalability and superior capability in preserving global image structures while maintaining fine details. To our best knowledge, $\infty$-Brush is the first conditional diffusion model in function space, that can controllably synthesize images at arbitrary resolutions of up to $4096\times4096$ pixels. The code is available at https://github.com/cvlab-stonybrook/infinity-brush.<|reference_end|>
@article{le2024$\infty$-brush:,
    title={$\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models
  in Infinite Dimensions},
    author={Minh-Quan Le, Alexandros Graikos, Srikar Yellapragada, Rajarsi Gupta,
  Joel Saltz, Dimitris Samaras},
    journal={arXiv preprint arXiv:2407.14709},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.14709},
    primaryClass={cs.CV}
}