Model Card for Initial Noise Loader for Stable Diffusion XL

This custom pipeline contains an initial noise loader class (class `` NoiseLoaderMixin’’ inspired from LoRA / textual inversion loaders in the diffusers library) for Stable Diffusion XL architecture. The initial noise loader allows to change the distribution of initial noise the generation process starts from with a single line of code “custom_pipeline.load_initial_noise_modifier(…)”. Currently implemented methods:

Start generation from a fixed noise. Example: custom_pipeline.load_initial_noise_modifier(method="fixed-seed", seed=…)
Golden Noise for Diffusion Models: A Learning Framework (Zhou et al., https://arxiv.org/abs/2411.09502). Example: custom_pipeline.load_initial_noise_modifier(method="golden-noise", npnet_path=…)

Citation

If you find my code useful, you may cite: @misc{initial_noise, author = {Syrine Noamen}, title = { Initial Noise Loader for Stable Diffusion XL - HuggingFace},

year = 2025, publisher = { HugginFace }, journal = { Hugging Face repository}, howpublished = {\url{https://huggingface.co/syrinenoamen/stable-diffusion_xl_initial_noise_loader}}, }

Example 1: Start generation from a fixed noise

This example is mostly for demonstration as this can already be achieved easily in the diffusers library.

Uses

from diffusers import DiffusionPipeline
custom_pipeline = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    variant="fp16",
    torch_dtype=torch.float16,
    use_safetensors=True,
    custom_pipeline="syr99/initial_noise_loader"
).to(device)
custom_pipeline.load_initial_noise_modifier(method="fixed-seed", seed=12345)

Example 2: Golden Noise for Diffusion Models: A Learning Framework (Zhou et al., https://arxiv.org/abs/2411.09502)

Requirements

pip install timm einops

Uses

from diffusers import DiffusionPipeline
custom_pipeline = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    variant="fp16",
    torch_dtype=torch.float16,
    use_safetensors=True,
    custom_pipeline="syrinenoamen/initial_noise_loader"
).to(device)

Citation Golden Noise

Code adapted from Github

@misc{zhou2024goldennoisediffusionmodels,
      title={Golden Noise for Diffusion Models: A Learning Framework}, 
      author={Zikai Zhou and Shitong Shao and Lichen Bai and Zhiqiang Xu and Bo Han and Zeke Xie},
      year={2024},
      eprint={2411.09502},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2411.09502}, 
}