readme: ---
  license: creativeml-openrail-m
  tags:
  - stable-diffusion
  - stable-diffusion-diffusers
  - text-to-image
  inference: false
  library_name: diffusers
  extra_gated_prompt: |-
    One more step before getting this model.
    This model is open access and available to all, with a CreativeML OpenRAIL-M license further specifying rights and usage.
    The CreativeML OpenRAIL License specifies: 
  
    1. You can't use the model to deliberately produce nor share illegal or harmful outputs or content 
    2. CompVis claims no rights on the outputs you generate, you are free to use them and are accountable for their use which must not go against the provisions set in the license
    3. You may re-distribute the weights and use the model commercially and/or as a service. If you do, please be aware you have to include the same use restrictions as the ones in the license and share a copy of the CreativeML OpenRAIL-M to all your users (please read the license entirely and carefully)
    Please read the full license here: https://huggingface.co/spaces/CompVis/stable-diffusion-license
    
    By clicking on "Access repository" below, you accept that your *contact information* (email address and username) can be shared with the model authors as well.
      
  extra_gated_fields:
   I have read the License and agree with its terms: checkbox
  ---
  
  Stable Diffusion Inpainting is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input, with the extra capability of inpainting the pictures by using a mask.
  
  The **Stable-Diffusion-Inpainting** was initialized with the weights of the [Stable-Diffusion-v-1-2](https://steps/huggingface.co/CompVis/stable-diffusion-v-1-2-original). First 595k steps regular training, then 440k steps of inpainting training at resolution 512x512 on “laion-aesthetics v2 5+” and 10% dropping of the text-conditioning to improve classifier-free [classifier-free guidance sampling](https://arxiv.org/abs/2207.12598). For inpainting, the UNet has 5 additional input channels (4 for the encoded masked-image and 1 for the mask itself) whose weights were zero-initialized after restoring the non-inpainting checkpoint. During training, we generate synthetic masks and in 25% mask everything.
  
  [![Open In Spaces](https://camo.githubusercontent.com/00380c35e60d6b04be65d3d94a58332be5cc93779f630bcdfc18ab9a3a7d3388/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f25463025394625413425393725323048756767696e67253230466163652d5370616365732d626c7565)](https://huggingface.co/spaces/runwayml/stable-diffusion-inpainting) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/in_painting_with_stable_diffusion_using_diffusers.ipynb)
   :-------------------------:|:-------------------------:|
  ## Examples:
  
  You can use this both with the [🧨Diffusers library](https://github.com/huggingface/diffusers) and the [RunwayML GitHub repository](https://github.com/runwayml/stable-diffusion).
  
  ### Diffusers
  
  ```python
  from diffusers import StableDiffusionInpaintPipeline
  
  pipe = StableDiffusionInpaintPipeline.from_pretrained(
      "runwayml/stable-diffusion-inpainting",
      revision="fp16",
      torch_dtype=torch.float16,
  )
  prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
  #image and mask_image should be PIL images.
  #The mask structure is white for inpainting and black for keeping as is
  image = pipe(prompt=prompt, image=image, mask_image=mask_image).images[0]
  image.save("./yellow_cat_on_park_bench.png")
  ```
  
  **How it works:**
  `image`          | `mask_image`
  :-------------------------:|:-------------------------:|
  <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" alt="drawing" width="300"/> | <img src="https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" alt="drawing" width="300"/>
  
  
  `prompt`          | `Output`
  :-------------------------:|:-------------------------:|
  <span style="position: relative;bottom: 150px;">Face of a yellow cat, high resolution, sitting on a park bench</span> | <img src="https://huggingface.co/datasets/patrickvonplaten/images/resolve/main/test.png" alt="drawing" width="300"/>
  
  ### Original GitHub Repository
  
  1. Download the weights [sd-v1-5-inpainting.ckpt](https://huggingface.co/runwayml/stable-diffusion-inpainting/resolve/main/sd-v1-5-inpainting.ckpt)
  2. Follow instructions [here](https://github.com/runwayml/stable-diffusion#inpainting-with-stable-diffusion).
  
  ## Model Details
  - **Developed by:** Robin Rombach, Patrick Esser
  - **Model type:** Diffusion-based text-to-image generation model
  - **Language(s):** English
  - **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license) is an [Open RAIL M license](https://www.licenses.ai/blog/2022/8/18/naming-convention-