CompVis

/

stable-diffusion-v-1-3-original

like 17

Follow

CompVis 1,121

I'm interested in this too, and opened this issue for further discussion: https://github.com/huggingface/diffusers/issues/208

I am also curious.

As far as I can tell, seed.py:30;seed_everything() is responsible for emplacing the seed at the beginning, but sampling permutes the seeding to a state which is, at the least, non-sequential (from the global seed), and additionally may not map to any global seed possible (non-obtainable, destructive permutation).

As you can see there is no repetition across sequential seeds on neither PLMS nor DDIM:

	Seed 41;	Seed 42
PLMS
DDIM

A further investigation hopefully will yield a way to make these seeds sequentially coherent. It may not be possible to sample multiple images per iteration with deterministic seeding, but it's possible that it might work per iteration. I'll try a few things, starting with re-seeding sequentially each iteration.

OK, yep, I was able to get row-determinism by re-seeding each iteration.

	Seed 42; n_iter=2	Seed 43 ; n_iter=2	Seed 42; n_iter=3

If you limit yourself to making just one image per call to txt2img.py, then there is no problem, I think. The problem arises from making multiple images and the way the code is structured.

Fixing the seed issue will require modifying txt2img.py. I have looked over the code, and there is one issue that makes solving the seeding issue challenging, and that is the use of batch size greater than 1 (n_samples, line 240). If you stick with batch size of 1, then it is fairly easy to re-seed before each call to the sampler. Note that n_iter can be greater than 1 without causing any issues.

Making the above modifications would be pretty straightforward, but I don't think we can expect the maintainers of the github to make such changes.

@spychiatrist

Nice! In the case that you want non-sequential random number seeds, you could use these lines instead:
import random
opt.seed = random.randrange(100000)

Good news! I have implemented a workaround for the non-determinism.

See this commit on my fork: https://github.com/spychiatrist/stable-diffusion-improvements/commit/255b6c14c48a6b868c17b2e7f39616d5adaad601

Here's a more detailed writeup with pretty pictures: sequential seeding (for n_iter) now with sub-seed offset (for n_samples). One new parameter --seed_offset allows you to deterministically shift which sample-subseed is selected.

Here's an example of deterministically shifting down and right in the matrix (previously impossible):

	--n_samples 3 --n_iter 3 --seed 42 --seed_offset 0	--n_samples 2 --n_iter 2 --seed 43 --seed_offset 1

See this commit (and priors) on my fork