readme: ---
  license: creativeml-openrail-m
  tags:
  - coreml
  - stable-diffusion
  - text-to-image
  ---
  
  # Core ML Converted Model
  
  This model was converted to Core ML for use on Apple Silicon devices by following Apple's instructions [here](https://github.com/apple/ml-stable-diffusion#-converting-models-to-core-ml).<br>
  Provide the model to an app such as [Mochi Diffusion](https://github.com/godly-devotion/MochiDiffusion) to generate images.<br>
  
  `split_einsum` version is compatible with all compute unit options including Neural Engine.<br>
  `original` version is only compatible with CPU & GPU option.
  
  # Stable Diffusion v1-5 Model Card
  
  Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input.
  For more information about how Stable Diffusion functions, please have a look at [🤗's Stable Diffusion blog](https://huggingface.co/blog/stable_diffusion).
  
  The **Stable-Diffusion-v1-5** checkpoint was initialized with the weights of the [Stable-Diffusion-v1-2](https:/steps/huggingface.co/CompVis/stable-diffusion-v1-2) 
  checkpoint and subsequently fine-tuned on 595k steps at resolution 512x512 on "laion-aesthetics v2 5+" and 10% dropping of the text-conditioning to improve [classifier-free guidance sampling](https://arxiv.org/abs/2207.12598).
  
  You can use this both with the [🧨Diffusers library](https://github.com/huggingface/diffusers) and the [RunwayML GitHub repository](https://github.com/runwayml/stable-diffusion).
  
  ### Diffusers
  ```py
  from diffusers import StableDiffusionPipeline
  import torch
  
  model_id = "runwayml/stable-diffusion-v1-5"
  pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
  pipe = pipe.to("cuda")
  
  prompt = "a photo of an astronaut riding a horse on mars"
  image = pipe(prompt).images[0]  
      
  image.save("astronaut_rides_horse.png")
  ```
  For more detailed instructions, use-cases and examples in JAX follow the instructions [here](https://github.com/huggingface/diffusers#text-to-image-generation-with-stable-diffusion)
  
  ### Original GitHub Repository
  
  1. Download the weights 
     - [v1-5-pruned-emaonly.ckpt](https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.ckpt) - 4.27GB, ema-only weight. uses less VRAM - suitable for inference
     - [v1-5-pruned.ckpt](https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned.ckpt) - 7.7GB, ema+non-ema weights. uses more VRAM - suitable for fine-tuning
  
  2. Follow instructions [here](https://github.com/runwayml/stable-diffusion).
  
  ## Model Details
  - **Developed by:** Robin Rombach, Patrick Esser
  - **Model type:** Diffusion-based text-to-image generation model
  - **Language(s):** English
  - **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license) is an [Open RAIL M license](https://www.licenses.ai/blog/2022/8/18/naming-convention-of-responsible-ai-licenses), adapted from the work that [BigScience](https://bigscience.huggingface.co/) and [the RAIL Initiative](https://www.licenses.ai/) are jointly carrying in the area of responsible AI licensing. See also [the article about the BLOOM Open RAIL license](https://bigscience.huggingface.co/blog/the-bigscience-rail-license) on which our license is based.
  - **Model Description:** This is a model that can be used to generate and modify images based on text prompts. It is a [Latent Diffusion Model](https://arxiv.org/abs/2112.10752) that uses a fixed, pretrained text encoder ([CLIP ViT-L/14](https://arxiv.org/abs/2103.00020)) as suggested in the [Imagen paper](https://arxiv.org/abs/2205.11487).
  - **Resources for more information:** [GitHub Repository](https://github.com/CompVis/stable-diffusion), [Paper](https://arxiv.org/abs/2112.10752).
  - **Cite as:**
  
        @InProceedings{Rombach_2022_CVPR,
            author    = {Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\"orn},
            title     = {High-Resolution Image Synthesis With Latent Diffusion Models},
            booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
            month     = {June},
            year      = {2022},
            pages     = {10684-10695}
        }
  
  # Uses
  
  ## Direct Use 
  The model is intended for research purposes only. Possible research areas and
  tasks include
  
  - Safe deployment of models which have the potential to generate harmful content.
  - Probing and understanding the limitations and biases of generative models.
  - Generation of artworks and use in design and other artistic processes.
  - Applications in educational or creative tools.
  - Research on generative models.
  
  Excluded uses are described below.
  
   ### Misuse, Malicious Use, and Out-of-Scope Use
  _Note: This section is taken from the [DALLE-MINI model card](https://huggingface.co/dalle-mini/dalle-mini), but applies in the same way to Stable Diffusion v1_.
  
  
  The model should not be used to intentionally create or dissemina