stabilityai

/

stable-diffusion-3-medium

like 4.56k

Follow

Stability AI 5,896

https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sd3.md

I use paperspace gradient and was able to get that working with a couple of small adjustments to the code they recommend in that readme. My script is below, if it's helpful to anyone (I acknowledge that there may be more proper/efficient ways to script this; I'm not a pro, just muddling through).
A few important notes about the script:

My gradient notebook was instantiated using the "cyberes/gradient-base-py3.10" image ... I have no idea how any of the following will behave on a vanilla paperspace notebook
You need to open Terminal and run hugging-cli login first, you also need gated access to the SD3 model here (duh)
You have to manually delete a file called /notebooks/storage/diffusers/examples/dreambooth/dog/.huggingface ... you may have to run the script once before this file is even formed, I'm not sure. However it happens, once it exists, it must be nuked for the script to run.
I couldn't get wandb to work, so I added a !wandb disabled line before accelerate gets run. I assume you can just remove the --report_to line from the training script at the end, if you prefer.
!pip install --upgrade diffusers[torch]
!pip install --upgrade diffusers[flax]
!pip install peft bitsandbytes

!git clone https://github.com/huggingface/diffusers
%cd diffusers
!pip install -e .

%cd examples/dreambooth

!pip install -r requirements_sd3.txt
!accelerate config default
#from accelerate.utils import write_basic_config
#write_basic_config()

from huggingface_hub import snapshot_download

local_dir = "./dog-ex"
snapshot_download(
"diffusers/dog-example",
local_dir=local_dir, repo_type="dataset",
ignore_patterns=".gitattributes",
)

MODEL_NAME="stabilityai/stable-diffusion-3-medium-diffusers"
INSTANCE_DIR="dog"
OUTPUT_DIR="trained-sd3-lora"

!wandb disabled

!accelerate launch train_dreambooth_lora_sd3.py
--pretrained_model_name_or_path=$MODEL_NAME
--instance_data_dir=$INSTANCE_DIR
--output_dir=/raid/.cache/$OUTPUT_DIR
--mixed_precision="fp16"
--instance_prompt="a photo of sks dog"
--resolution=1024
--train_batch_size=1
--gradient_accumulation_steps=4
--learning_rate=1e-5
--report_to="wandb"
--lr_scheduler="constant"
--lr_warmup_steps=0
--max_train_steps=500
--weighting_scheme="logit_normal"
--validation_prompt="A photo of sks dog in a bucket"
--validation_epochs=25
--seed="0"
--push_to_hub