Flux Anime Landscape lora Usage

🙂 This lora focus light up the anime landscape scene. ☀
Trigger word: anime style

Installtion

pip install -U diffusers transformers torch sentencepiece peft controlnet-aux moviepy protobuf

Demo

1 Whimsical Wildlife: Red Panda & Bird Ballet

import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)
pipe.load_lora_weights("Flux_Anime_Landscape_Lora/my_first_flux_lora_v1_000001500.safetensors")
pipe.enable_sequential_cpu_offload()

prompt = "anime style ,high quality nature video featuring a red panda balancing on a bamboo stem while a bird lands on it's head, on the background there is a waterfall"

image = pipe(prompt,
             num_inference_steps=50,
             guidance_scale=3.5,
            ).images[0]
image.save("ani0.png")
from IPython import display
display.Image("ani0.png", width=512, height=512)

2 Tranquil Reverie: Blossoms & Books

prompt = "anime style, Sunlight filters through clouds onto a cherry blossom-filled meadow, where a blue-haired youth in an indigo coat stands by a vermilion bridge, holding a book as petals drift past his smile. The scene blends dynamic nature with human stillness, using pink, red and blue hues to create harmony, while symbolic elements like blossoms and books add depth."

image = pipe(prompt,
             num_inference_steps=50,
             guidance_scale=3.5,
            ).images[0]
image.save("ani1.png")
from IPython import display
display.Image("ani1.png", width=512, height=512)

3 Train

anime style ,This digital illustration in a vibrant, anime-inspired style depicts a vintage green tram with "P.R. 13" displayed on its front, traveling down a wet, reflective railway track during a stunning sunset. The sky is ablaze with vivid orange, pink, and purple clouds, casting a warm glow over the scene. On the left, a streetlight and power poles line the track, while small buildings and a few trees are visible on the right. The tram's headlights and windows reflect the colorful sky, and the tracks glisten with rain. The overall mood is nostalgic and serene, capturing a picturesque urban sunset.

OmniConsistency Demo

1 Contemplative Contrast: Scholar in the Greenery

refer to https://github.com/showlab/OmniConsistency
Image Input (xiang_image.jpg)

import time
import torch
import os
from tqdm import tqdm
from PIL import Image
from src_inference.pipeline import FluxPipeline
from src_inference.lora_helper import set_single_lora

# 初始化基础模型
base_path = "black-forest-labs/FLUX.1-dev"
pipe = FluxPipeline.from_pretrained(base_path, torch_dtype=torch.bfloat16)

# 设置OmniConsistency LoRA
set_single_lora(pipe.transformer, 
                "OmniConsistency/OmniConsistency.safetensors", 
                lora_weights=[1], cond_size=512)

pipe.load_lora_weights("Flux_Anime_Landscape_Lora/my_first_flux_lora_v1_000001500.safetensors")

# 启用CPU offload节省显存
pipe.enable_sequential_cpu_offload()

image_path1 = "xiang_image.jpg"

subject_images = []
spatial_image = [Image.open(image_path1).convert("RGB")]

width, height = 1024, 1024

style_prompt = "anime style, A young man with black hair and glasses sits outdoors in a white T-shirt, surrounded by lush green foliage. The natural backdrop contrasts with his relaxed posture, creating a serene vibe."

image = pipe(
        style_prompt,
        height=height,
        width=width,
        guidance_scale=3.5,
        num_inference_steps=25,
        max_sequence_length=512,
        generator=torch.Generator("cpu").manual_seed(5),
        spatial_images=spatial_image,
        subject_images=subject_images,
        cond_size=512,
    ).images[0]

image.save("im0.png")
from IPython import display
display.Image("im0.png", width=512, height=512)