readme: ---
  license: creativeml-openrail-m
  thumbnail: "https://huggingface.co/Linaqruf/hitokomoru-diffusion-v2/resolve/main/example_image/thumbnail.png"
  language:
  - en
  pipeline_tag: text-to-image
  tags:
  - stable-diffusion
  - stable-diffusion-diffusers
  - diffusers
  - waifu-diffusion
  inference: true
  widget:
  - text: >-
      masterpiece, best quality, 1girl, brown hair, green eyes, colorful, autumn,
      cumulonimbus clouds, lighting, blue sky, falling leaves, garden
    example_title: example 1girl
  - text: >-
      masterpiece, best quality, 1boy, medium hair, blonde hair, blue eyes,
      bishounen, colorful, autumn, cumulonimbus clouds, lighting, blue sky,
      falling leaves, garden
    example_title: example 1boy
  ---
  
  # Hitokomoru Diffusion V2
  
  ![Anime Girl](https://huggingface.co/Linaqruf/hitokomoru-diffusion-v2/resolve/main/example_image/thumbnail.png)
  
  A latent diffusion model that has been trained on Japanese Artist artwork, [ヒトこもる/Hitokomoru](https://www.pixiv.net/en/users/30837811). The current model is fine-tuned from [waifu-diffusion-1-4](https://huggingface.co/hakurei/waifu-diffusion-v1-4) (`wd-1-4-anime_e2.ckpt`) with a learning rate of `2.0e-6`, 15000 training steps and 4 batch sizes on the `257 artworks` collected from Danbooru. This model supposed to be a continuation of [hitokomoru-diffusion](https://huggingface.co/Linaqruf/hitokomoru-diffusion/) fine-tuned from Anything V3.0. Dataset has been preprocessed using [Aspect Ratio Bucketing Tool](https://github.com/NovelAI/novelai-aspect-ratio-bucketing) so that it can be converted to latents and trained at non-square resolutions. Like other anime-style Stable Diffusion models, it also supports Danbooru tags to generate images.
  
  e.g. **_1girl, white hair, golden eyes, beautiful eyes, detail, flower meadow, cumulonimbus clouds, lighting, detailed sky, garden_** 
  
  - Use it with the [`Automatic1111's Stable Diffusion Webui`](https://github.com/AUTOMATIC1111/stable-diffusion-webui) see: [how-to-use](#howtouse)
  - Use it with 🧨 [`diffusers`](##🧨Diffusers)
  
  
  # Model Details
  
  - **Developed by:** Linaqruf
  - **Model type:** Diffusion-based text-to-image generation model
  - **Model type:** This is a model that can be used to generate and modify images based on text prompts. 
  - **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/LICENSE-MODEL)
  - **Finetuned from model:** [waifu-diffusion-v1-4-epoch-2](https://huggingface.co/hakurei/waifu-diffusion-v1-4/blob/main/wd-1-4-anime_e2.ckpt)
  
  ## How to Use
  - Download the `hitokomoru-v2.ckpt` [here](https://huggingface.co/Linaqruf/hitokomoru-diffusion-v2/resolve/main/hitokomoru-v2.ckpt), or download the safetensors version [here](https://huggingface.co/Linaqruf/hitokomoru-diffusion-v2/resolve/main/hitokomoru-v2.safetensors).
  - This model is fine-tuned from [waifu-diffusion-v1-4-epoch-2](https://huggingface.co/hakurei/waifu-diffusion-v1-4/blob/main/wd-1-4-anime_e2.ckpt), which is also fine-tuned from [stable-diffusion-2-1-base](https://huggingface.co/stabilityai/stable-diffusion-2-1-base). So in order to run this model in [`Automatic1111's Stable Diffusion Webui`](https://github.com/AUTOMATIC1111/stable-diffusion-webui), you need to put inference config .YAML file next to the model, you can find it [here](https://huggingface.co/Linaqruf/hitokomoru-diffusion-v2/resolve/main/hitokomoru-v2.yaml)
  - You need to adjust your prompt using aesthetic tags, Based [Official Waifu Diffusion 1.4 release notes](https://gist.github.com/harubaru/8581e780a1cf61352a739f2ec2eef09b#prompting), an ideal negative prompt to guide the model towards high aesthetic generations would look like:
  ```
  worst quality, low quality, medium quality, deleted, lowres, comic, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry
  ```
  - And, the following should also be prepended to prompts to get high aesthetic results:
  ```
  masterpiece, best quality, high quality, absurdres
  ```
  ## 🧨 Diffusers
  
  This model can be used just like any other Stable Diffusion model. For more information, please have a look at the [Stable Diffusion](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion). You can also export the model to [ONNX](https://huggingface.co/docs/diffusers/optimization/onnx), [MPS](https://huggingface.co/docs/diffusers/optimization/mps) and/or [FLAX/JAX]().
  
  You should install dependencies below in order to running the pipeline
  
  ```bash
  pip install diffusers transformers accelerate scipy safetensors
  ```
  Running the pipeline (if you don't swap the scheduler it will run with the default DDIM, in this example we are swapping it to DPMSolverMultistepScheduler):
  
  ```python
  import torch
  from torch import autocast
  from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler
  
  model_id = "Linaqruf/hitokomoru-diffusion-v2"
  
  # Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
  pipe = StableDi