wan2.1 1.3b Lora - ValueError: Invalid LoRA checkpoint.
all 1.3b model failing , can you suggest you model works with Wan-AI/Wan2.1-T2V-1.3B-Diffusers or which one to use
for exp I tried this
LORA_REPO = "Kijai/WanVideo_comfy"
LORA_FILE = "Wan2_1-T2V-1_3B_bf16.safetensors"
MODEL_ID = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
That's a full model not a Lora, and it's targeted at Comfy, what are you trying to do?
I am trying to run via gradio
import torch
from diffusers import AutoencoderKLWan, WanPipeline
from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler
from diffusers.utils import export_to_video
import gradio as gr
import spaces
LORA_REPO = "Kijai/WanVideo_comfy"
LORA_FILE = "Wan2_1-T2V-1_3B_bf16.safetensors"
MODEL_ID = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
LORA_SCALE = 1.0
FLOW_SHIFT = 7
EMBEDDED_GUIDANCE_SCALE = 6.0
=== Load model and VAE ===
vae = AutoencoderKLWan.from_pretrained(MODEL_ID, subfolder="vae", torch_dtype=torch.float32)
pipe = WanPipeline.from_pretrained(
MODEL_ID,
vae=vae,
torch_dtype=torch.bfloat16
)
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config, flow_shift=FLOW_SHIFT)
pipe.to("cuda")
=== Load LoRA weights ====
pipe.load_lora_weights(
LORA_REPO,
weight_name=LORA_FILE,
lora_scale=LORA_SCALE,
lora_alpha=0.7 # matches your adapter config
)
pipe.enable_model_cpu_offload()
=== Default Negative Prompt ===
default_negative_prompt = (
"Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, "
)
=== Video generation ===
@spaces.GPU(duration=100)
def generate(prompt, negative_prompt, guidance_scale, height, width, num_frames):
output = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
height=height,
width=width,
num_frames=num_frames,
guidance_scale=guidance_scale + EMBEDDED_GUIDANCE_SCALE # embedding boost
).frames[0]
output_path = "output.mp4"
export_to_video(output, output_path, fps=16)
return output_path
=== Gradio UI ===
gr.Interface(
fn=generate,
inputs=[
gr.Textbox(label="Prompt", value="Low speed, documentary style picture, a lively white puppy is running quickly on the green grass..."),
gr.Textbox(label="Negative Prompt", value=default_negative_prompt, lines=4),
gr.Slider(label="Guidance Scale", minimum=1.0, maximum=10.0, value=5.0),
gr.Slider(label="Height", minimum=256, maximum=1024, step=64, value=480),
gr.Slider(label="Width", minimum=256, maximum=1024, step=64, value=832),
gr.Slider(label="Number of Frames", minimum=16, maximum=128, step=1, value=81)
],
outputs=gr.Video(label="Generated Video"),
title="Wan 2.1 T2V - Rahul7star LoRA v1",
description="Text-to-video generation using Wan 2.1 and custom LoRA"
).launch()