SD-$\pi$XL: Generating Low-Resolution Quantized Imagery via Score Distillation
<|reference_start|>SD-$\pi$XL: Generating Low-Resolution Quantized Imagery via Score Distillation: Low-resolution quantized imagery, such as pixel art, is seeing a revival in modern applications ranging from video game graphics to digital design and fabrication, where creativity is often bound by a limited palette of elemental units. Despite their growing popularity, the automated generation of quantized images from raw inputs remains a significant challenge, often necessitating intensive manual input. We introduce SD-$\pi$XL, an approach for producing quantized images that employs score distillation sampling in conjunction with a differentiable image generator. Our method enables users to input a prompt and optionally an image for spatial conditioning, set any desired output size $H \times W$, and choose a palette of $n$ colors or elements. Each color corresponds to a distinct class for our generator, which operates on an $H \times W \times n$ tensor. We adopt a softmax approach, computing a convex sum of elements, thus rendering the process differentiable and amenable to backpropagation. We show that employing Gumbel-softmax reparameterization allows for crisp pixel art effects. Unique to our method is the ability to transform input images into low-resolution, quantized versions while retaining their key semantic features. Our experiments validate SD-$\pi$XL's performance in creating visually pleasing and faithful representations, consistently outperforming the current state-of-the-art. Furthermore, we showcase SD-$\pi$XL's practical utility in fabrication through its applications in interlocking brick mosaic, beading and embroidery design.<|reference_end|>
@article{binninger2024sd-$\pi$xl:,
    title={SD-$\pi$XL: Generating Low-Resolution Quantized Imagery via Score
  Distillation},
    author={Alexandre Binninger and Olga Sorkine-Hornung},
    journal={arXiv preprint arXiv:2410.06236},
    year={2024},
    doi={10.1145/3680528.3687570},
    archivePrefix={arXiv},
    eprint={2410.06236},
    primaryClass={cs.CV cs.GR}
}