readme: ---
  license: openrail++
  tags:
  - stable-diffusion
  - text-to-image
  pinned: true
  ---
  
  # Model Card for flex-diffusion-2-1
  
  <!-- Provide a quick summary of what the model is/does. [Optional] -->
  stable-diffusion-2-1 (stabilityai/stable-diffusion-2-1) finetuned with different aspect ratios.
  
  ## TLDR:
  
  ### There are 2 models in this repo:
  - One based on stable-diffusion-2-1 (stabilityai/stable-diffusion-2-1) finetuned for 6k steps.
  - One based on stable-diffusion-2-base (stabilityai/stable-diffusion-2-base) finetuned for 6k steps, on the same dataset.
  
  For usage, see - [How to Get Started with the Model](#how-to-get-started-with-the-model)
  
  ### It aims to solve the following issues:
  1. Generated images looks like they are cropped from a larger image.
  
  2. Generating non-square images creates weird results, due to the model being trained on square images.
  Examples:
  
  | resolution      | model   |   stable diffusion           |   flex diffusion              |
  |:---------------:|:-------:|:----------------------------:|:-----------------------------:|
  | 576x1024 (9:16) | v2-1    | ![img](imgs/21-576-1024.png) | ![img](imgs/21f-576-1024.png) |
  | 576x1024 (9:16) | v2-base | ![img](imgs/2b-576-1024.png) | ![img](imgs/2bf-576-1024.png) |
  | 1024x576 (16:9) | v2-1    | ![img](imgs/21-1024-576.png) | ![img](imgs/21f-1024-576.png) |
  | 1024x576 (16:9) | v2-base | ![img](imgs/2b-1024-576.png) | ![img](imgs/2bf-1024-576.png) |
  
  ### Limitations:
  1. It's trained on a small dataset, so it's improvements may be limited.
  2. For each aspect ratio, it's trained on only a fixed resolution. So it may not be able to generate images of different resolutions.
  For 1:1 aspect ratio, it's fine-tuned at 512x512, although flex-diffusion-2-1 was last finetuned at 768x768.
  
  ### Potential improvements:
  1. Train on a larger dataset.
  2. Train on different resolutions even for the same aspect ratio.
  3. Train on specific aspect ratios, instead of a range of aspect ratios.
  
  
  #  Table of Contents
  
  - [Model Card for flex-diffusion-2-1](#model-card-for--model_id-)
  - [Table of Contents](#table-of-contents)
  - [Table of Contents](#table-of-contents-1)
  - [Model Details](#model-details)
    - [Model Description](#model-description)
  - [Uses](#uses)
    - [Direct Use](#direct-use)
    - [Downstream Use [Optional]](#downstream-use-optional)
    - [Out-of-Scope Use](#out-of-scope-use)
  - [Bias, Risks, and Limitations](#bias-risks-and-limitations)
    - [Recommendations](#recommendations)
  - [Training Details](#training-details)
    - [Training Data](#training-data)
    - [Training Procedure](#training-procedure)
      - [Preprocessing](#preprocessing)
      - [Speeds, Sizes, Times](#speeds-sizes-times)
  - [Evaluation](#evaluation)
    - [Testing Data, Factors & Metrics](#testing-data-factors--metrics)
      - [Testing Data](#testing-data)
      - [Factors](#factors)
      - [Metrics](#metrics)
    - [Results](#results)
  - [Model Examination](#model-examination)
  - [Environmental Impact](#environmental-impact)
  - [Technical Specifications [optional]](#technical-specifications-optional)
    - [Model Architecture and Objective](#model-architecture-and-objective)
    - [Compute Infrastructure](#compute-infrastructure)
      - [Hardware](#hardware)
      - [Software](#software)
  - [Citation](#citation)
  - [Glossary [optional]](#glossary-optional)
  - [More Information [optional]](#more-information-optional)
  - [Model Card Authors [optional]](#model-card-authors-optional)
  - [Model Card Contact](#model-card-contact)
  - [How to Get Started with the Model](#how-to-get-started-with-the-model)
  
  
  # Model Details
  
  ## Model Description
  
  <!-- Provide a longer summary of what this model is/does. -->
  stable-diffusion-2-1 (stabilityai/stable-diffusion-2-1) finetuned for dynamic aspect ratios.
  
  finetuned resolutions:
  |    |   width |   height | aspect ratio   |
  |---:|--------:|---------:|:---------------|
  |  0 |     512 |     1024 | 1:2            |
  |  1 |     576 |     1024 | 9:16           |
  |  2 |     576 |      960 | 3:5            |
  |  3 |     640 |     1024 | 5:8            |
  |  4 |     512 |      768 | 2:3            |
  |  5 |     640 |      896 | 5:7            |
  |  6 |     576 |      768 | 3:4            |
  |  7 |     512 |      640 | 4:5            |
  |  8 |     640 |      768 | 5:6            |
  |  9 |     640 |      704 | 10:11          |
  | 10 |     512 |      512 | 1:1            |
  | 11 |     704 |      640 | 11:10          |
  | 12 |     768 |      640 | 6:5            |
  | 13 |     640 |      512 | 5:4            |
  | 14 |     768 |      576 | 4:3            |
  | 15 |     896 |      640 | 7:5            |
  | 16 |     768 |      512 | 3:2            |
  | 17 |    1024 |      640 | 8:5            |
  | 18 |     960 |      576 | 5:3            |
  | 19 |    1024 |      576 | 16:9           |
  | 20 |    1024 |      512 | 2:1            |
  
  - **Developed by:** Jonathan Chang
  - **Model type:** Diffusion-based text-to-image generation model
  - **Language(s)**: English
  - **License:** creativeml-openrail-m
  - **Parent Model:** https://huggingfa