dn6 HF staff commited on
Commit
e6b0429
1 Parent(s): 74c1349

update README

Browse files
Files changed (1) hide show
  1. README.md +23 -21
README.md CHANGED
@@ -3,6 +3,8 @@ pipeline_tag: text-to-image
3
  license: other
4
  license_name: stable-cascade-nc-community
5
  license_link: LICENSE
 
 
6
  ---
7
 
8
  # Stable Cascade
@@ -10,13 +12,13 @@ license_link: LICENSE
10
  <!-- Provide a quick summary of what the model is/does. -->
11
  <img src="figures/collage_1.jpg" width="800">
12
 
13
- This model is built upon the [Würstchen](https://openreview.net/forum?id=gU58d5QeGv) architecture and its main
14
- difference to other models like Stable Diffusion is that it is working at a much smaller latent space. Why is this
15
- important? The smaller the latent space, the **faster** you can run inference and the **cheaper** the training becomes.
16
- How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being
17
- encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a
18
- 1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the
19
- highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable
20
  Diffusion 1.5. <br> <br>
21
  Therefore, this kind of model is well suited for usages where efficiency is important. Furthermore, all known extensions
22
  like finetuning, LoRA, ControlNet, IP-Adapter, LCM etc. are possible with this method as well.
@@ -41,27 +43,27 @@ For research purposes, we recommend our `StableCascade` Github repository (https
41
  ### Model Overview
42
  Stable Cascade consists of three models: Stage A, Stage B and Stage C, representing a cascade to generate images,
43
  hence the name "Stable Cascade".
44
- Stage A & B are used to compress images, similar to what the job of the VAE is in Stable Diffusion.
45
- However, with this setup, a much higher compression of images can be achieved. While the Stable Diffusion models use a
46
- spatial compression factor of 8, encoding an image with resolution of 1024 x 1024 to 128 x 128, Stable Cascade achieves
47
- a compression factor of 42. This encodes a 1024 x 1024 image to 24 x 24, while being able to accurately decode the
48
- image. This comes with the great benefit of cheaper training and inference. Furthermore, Stage C is responsible
49
  for generating the small 24 x 24 latents given a text prompt. The following picture shows this visually.
50
 
51
  <img src="figures/model-overview.jpg" width="600">
52
 
53
- For this release, we are providing two checkpoints for Stage C, two for Stage B and one for Stage A. Stage C comes with
54
- a 1 billion and 3.6 billion parameter version, but we highly recommend using the 3.6 billion version, as most work was
55
- put into its finetuning. The two versions for Stage B amount to 700 million and 1.5 billion parameters. Both achieve
56
- great results, however the 1.5 billion excels at reconstructing small and fine details. Therefore, you will achieve the
57
- best results if you use the larger variant of each. Lastly, Stage A contains 20 million parameters and is fixed due to
58
  its small size.
59
 
60
  ## Evaluation
61
  <img height="300" src="figures/comparison.png"/>
62
- According to our evaluation, Stable Cascade performs best in both prompt alignment and aesthetic quality in almost all
63
- comparisons. The above picture shows the results from a human evaluation using a mix of parti-prompts (link) and
64
- aesthetic prompts. Specifically, Stable Cascade (30 inference steps) was compared against Playground v2 (50 inference
65
  steps), SDXL (50 inference steps), SDXL Turbo (1 inference step) and Würstchen v2 (30 inference steps).
66
 
67
  ## Code Example
@@ -118,7 +120,7 @@ Excluded uses are described below.
118
 
119
  ### Out-of-Scope Use
120
 
121
- The model was not trained to be factual or true representations of people or events,
122
  and therefore using the model to generate such content is out-of-scope for the abilities of this model.
123
  The model should not be used in any way that violates Stability AI's [Acceptable Use Policy](https://stability.ai/use-policy).
124
 
 
3
  license: other
4
  license_name: stable-cascade-nc-community
5
  license_link: LICENSE
6
+ prior:
7
+ - stabilityai/stable-cascade-prior
8
  ---
9
 
10
  # Stable Cascade
 
12
  <!-- Provide a quick summary of what the model is/does. -->
13
  <img src="figures/collage_1.jpg" width="800">
14
 
15
+ This model is built upon the [Würstchen](https://openreview.net/forum?id=gU58d5QeGv) architecture and its main
16
+ difference to other models like Stable Diffusion is that it is working at a much smaller latent space. Why is this
17
+ important? The smaller the latent space, the **faster** you can run inference and the **cheaper** the training becomes.
18
+ How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being
19
+ encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a
20
+ 1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the
21
+ highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable
22
  Diffusion 1.5. <br> <br>
23
  Therefore, this kind of model is well suited for usages where efficiency is important. Furthermore, all known extensions
24
  like finetuning, LoRA, ControlNet, IP-Adapter, LCM etc. are possible with this method as well.
 
43
  ### Model Overview
44
  Stable Cascade consists of three models: Stage A, Stage B and Stage C, representing a cascade to generate images,
45
  hence the name "Stable Cascade".
46
+ Stage A & B are used to compress images, similar to what the job of the VAE is in Stable Diffusion.
47
+ However, with this setup, a much higher compression of images can be achieved. While the Stable Diffusion models use a
48
+ spatial compression factor of 8, encoding an image with resolution of 1024 x 1024 to 128 x 128, Stable Cascade achieves
49
+ a compression factor of 42. This encodes a 1024 x 1024 image to 24 x 24, while being able to accurately decode the
50
+ image. This comes with the great benefit of cheaper training and inference. Furthermore, Stage C is responsible
51
  for generating the small 24 x 24 latents given a text prompt. The following picture shows this visually.
52
 
53
  <img src="figures/model-overview.jpg" width="600">
54
 
55
+ For this release, we are providing two checkpoints for Stage C, two for Stage B and one for Stage A. Stage C comes with
56
+ a 1 billion and 3.6 billion parameter version, but we highly recommend using the 3.6 billion version, as most work was
57
+ put into its finetuning. The two versions for Stage B amount to 700 million and 1.5 billion parameters. Both achieve
58
+ great results, however the 1.5 billion excels at reconstructing small and fine details. Therefore, you will achieve the
59
+ best results if you use the larger variant of each. Lastly, Stage A contains 20 million parameters and is fixed due to
60
  its small size.
61
 
62
  ## Evaluation
63
  <img height="300" src="figures/comparison.png"/>
64
+ According to our evaluation, Stable Cascade performs best in both prompt alignment and aesthetic quality in almost all
65
+ comparisons. The above picture shows the results from a human evaluation using a mix of parti-prompts (link) and
66
+ aesthetic prompts. Specifically, Stable Cascade (30 inference steps) was compared against Playground v2 (50 inference
67
  steps), SDXL (50 inference steps), SDXL Turbo (1 inference step) and Würstchen v2 (30 inference steps).
68
 
69
  ## Code Example
 
120
 
121
  ### Out-of-Scope Use
122
 
123
+ The model was not trained to be factual or true representations of people or events,
124
  and therefore using the model to generate such content is out-of-scope for the abilities of this model.
125
  The model should not be used in any way that violates Stability AI's [Acceptable Use Policy](https://stability.ai/use-policy).
126