metadata

library_name: diffusers
license: openrail++
datasets:
  - ShinoharaHare/Danbooru-2024-Filtered-1M
language:
  - en
base_model:
  - ShinoharaHare/WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction
pipeline_tag: image-to-image
tags:
  - anime
  - art
  - inpaint
  - inpainting
  - stable-diffusion
  - stable-diffusion-xl

Waifu-Inpaint-XL

Waifu-Inpaint-XL is an SDXL-based model specifically designed for inpainting anime-themed images.

Task	Model
Generation	WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction
Inpainting	Waifu-Inpaint-XL
Colorizing	Waifu-Colorize-XL

Overview

Waifu-Inpaint-XL is initialized from WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction, with its input channels expanded from 4 to 9. Among these, 1 additional channel is dedicated to the mask, and 4 extra channels are used for masked latents.

Model Details

Developed by: ShinoharaHare
Model type: Diffusion based text-to-image/image-to-image generative model for inpainting
Language(s) (NLP): English
License: CreativeML Open RAIL++-M
Finetuned from: WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction

🧨 Diffusers

Image Mask Inpainted

import torch
from diffusers import StableDiffusionXLInpaintPipeline
from diffusers.utils import load_image

pipeline = StableDiffusionXLInpaintPipeline.from_pretrained(
  'ShinoharaHare/Waifu-Inpaint-XL',
  torch_dtype=torch.half
)
pipeline.to('cuda')

image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/tPo5oPJQpxWamM-tGIYqj.png')
mask_image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/QpmzmgROUM0eP53Cxx2Ih.png', lambda x: x.convert('L'))

inpainted_image = pipeline(
  prompt='blue eyes, holding red spider lily in hand',
  image=image,
  mask_image=mask_image,
  num_inference_steps=28,
  guidance_scale=5.0,
  height=image.height,
  width=image.width,
  generator=torch.Generator(pipeline.device).manual_seed(5)
).images[0]

inpainted_image.show()