File size: 3,824 Bytes
bff4af5 a11180f c1583da a11180f bff4af5 a11180f bff4af5 c1583da bff4af5 c1583da |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
---
license: apache-2.0
language:
- en
pipeline_tag: text-to-image
library_name: transformers
widget:
- text: |
A banana on the left of an apple.
base_model:
- Tencent-Hunyuan/HunyuanDiT
- Lykon/DreamShaper
- stabilityai/stable-diffusion-xl-base-1.0
---
# NPNet Pipeline Usage Guide😄
## Overview
This guide provides instructions on how to use the NPNet, a noise prompt network aims to transform the random Gaussian noise into golden noise, by adding a small desirable perturbation derived from the text prompt to boost the overall quality and semantic faithfulness of the synthesized images.
Here we provide the inference code which supports different models like ***Stable Diffusion XL, DreamShaper-xl-v2-turbo, and Hunyuan-DiT.***
## Requirements
- `python >= 3.8.0`
- `pytorch with cuda version`
- `diffusers`
- `PIL`
- `numpy`
- `timm`
- `argparse`
- `einops`
## Installation🚀️
Make sure you have successfully built `python` environment and installed `pytorch` with cuda version. Before running the script, ensure you have all the required packages installed. You can install them using:
```bash
pip install diffusers, PIL, numpy, timm, argparse, einops
```
## Usage👀️
To use the NPNet pipeline, you need to run the `npnet_pipeline.py` script with appropriate command-line arguments. Below are the available options:
### Command-Line Arguments
- `--pipeline`: Select the model pipeline (`SDXL`, `DreamShaper`, `DiT`). Default is `SDXL`.
- `--prompt`: The textual prompt based on which the image will be generated. Default is "A banana on the left of an apple."
- `--inference-step`: Number of inference steps for the diffusion process. Default is 50.
- `--cfg`: Classifier-free guidance scale. Default is 5.5.
- `--pretrained-path`: Path to the pretrained model weights. Default is a specified path in the script.
- `--size`: The size (height and width) of the generated image. Default is 1024.
### Running the Script
Run the script from the command line by navigating to the directory containing `npnet_pipeline.py` and executing:
```
python npnet_pipeline.py --pipeline SDXL --prompt "A banana on the left of an apple." --size 1024
```
This command will generate an image based on the prompt "A banana on the left of an apple." using the Stable Diffusion XL model with an image size of 1024x1024 pixels.
### Output🎉️
The script will save two images:
- A standard image generated by the diffusion model.
- A golden image generated by the diffusion model with the NPNet.
Both images will be saved in the current directory with names based on the model and prompt.
## Pre-trained Weights Download❤️
We provide the pre-trained NPNet weights of Stable Diffusion XL, DreamShaper-xl-v2-turbo, and Hunyuan-DiT with [google drive](https://drive.google.com/drive/folders/1Z0wg4HADhpgrztyT3eWijPbJJN5Y2jQt?usp=drive_link)
## Citation:
If you find our code useful for your research, please cite our paper.
```
@misc{zhou2024goldennoisediffusionmodels,
title={Golden Noise for Diffusion Models: A Learning Framework},
author={Zikai Zhou and Shitong Shao and Lichen Bai and Zhiqiang Xu and Bo Han and Zeke Xie},
year={2024},
eprint={2411.09502},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2411.09502},
}
```
## 🙏 Acknowledgements
We thank the community and contributors for their invaluable support in developing NPNet.
We thank @DataCTE for constructing the ComfyUI of NPNet inference code [ComfyUI](https://github.com/DataCTE/ComfyUI_Golden-Noise).
We thank @asagi4 for constructing the ComfyUI of NPNet inference code [ComfyUI](https://github.com/asagi4/ComfyUI-NPNet).
--- |