Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
base_model:
|
6 |
+
- Wan-AI/Wan2.1-T2V-14B
|
7 |
+
pipeline_tag: text-to-video
|
8 |
+
tags:
|
9 |
+
- text-to-video
|
10 |
+
- text-to-image
|
11 |
+
- lora
|
12 |
+
- diffusers
|
13 |
+
- template:diffusion-lora
|
14 |
+
widget:
|
15 |
+
- text: >-
|
16 |
+
abandoned places A steady zoom-out tall abandoned building covered in vines and trees stands in the middle of an abandoned city. The sky is overcast and the air is thick with fog. The city is mostly obscured by the fog, but you can see some other buildings in the distance. There is a sense of decay and abandonment in the image.
|
17 |
+
output:
|
18 |
+
url: example_videos/1.mp4
|
19 |
+
- text: >-
|
20 |
+
abandoned places A steady zoom-in to An old, rusty car, partially obscured by vegetation, sits in the middle of a dense forest. The car's body is a faded light blue color, with a rusted hood and roof. The car's door is open and the interior is dark and dusty. The windows are all broken and the seats are torn. The car is surrounded by tall trees and thick bushes. The ground is covered in leaves and debris. The overall atmosphere is one of decay and neglect.
|
21 |
+
output:
|
22 |
+
url: example_videos/2.mp4
|
23 |
+
- text: >-
|
24 |
+
abandoned places A grand, abandoned mansion, with peeling paint and broken windows, stands in a grove of overgrown trees. The front steps are crumbling and the driveway is cracked. The sky is overcast.
|
25 |
+
output:
|
26 |
+
url: example_videos/3.mp4
|
27 |
+
- text: >-
|
28 |
+
abandoned places A steady zoom-out from the center of an abandoned industrial complex, where a single rusted smokestack stands tall against the cloudy sky. The camera moves back, revealing a network of crumbling buildings and broken glass windows. The sound of distant thunder rumbles as the wind kicks up dust and loose debris.
|
29 |
+
output:
|
30 |
+
url: example_videos/4.mp4
|
31 |
+
---
|
32 |
+
|
33 |
+
<div style="background-color: #f8f9fa; padding: 20px; border-radius: 10px; margin-bottom: 20px;">
|
34 |
+
<h1 style="color: #24292e; margin-top: 0;">Abandoned Places LoRA for Wan2.1 14B T2V</h1>
|
35 |
+
|
36 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
37 |
+
<h2 style="color: #24292e; margin-top: 0;">Overview</h2>
|
38 |
+
<p>This LoRA is trained on the Wan2.1 14B T2V model and allows you to generate videos of abandoned places!</p>
|
39 |
+
</div>
|
40 |
+
|
41 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
42 |
+
<h2 style="color: #24292e; margin-top: 0;">Features</h2>
|
43 |
+
<ul style="margin-bottom: 0;">
|
44 |
+
<li>Trained on the Wan2.1 14B T2V base model</li>
|
45 |
+
<li>Consistent results across different object and scenes types</li>
|
46 |
+
<li>Simple prompt examples that are easy to adapt</li>
|
47 |
+
</ul>
|
48 |
+
</div>
|
49 |
+
|
50 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
51 |
+
<h2 style="color: #24292e; margin-top: 0;">Community</h2>
|
52 |
+
<ul style="margin-bottom: 0;">
|
53 |
+
<li><b>Discord:</b> <a href="https://remade.ai/join-discord?utm_source=Huggingface&utm_medium=Social&utm_campaign=model_release&utm_content=abandoned_places" style="color: #0366d6; text-decoration: none;">Join our community</a> to generate videos with this LoRA for free</li>
|
54 |
+
<li><b>Request LoRAs:</b> We're training and open-sourcing Wan2.1 LoRAs for free - join our Discord to make requests!</li>
|
55 |
+
</ul>
|
56 |
+
</div>
|
57 |
+
</div>
|
58 |
+
|
59 |
+
<Gallery />
|
60 |
+
|
61 |
+
# Model File and Inference Workflow
|
62 |
+
|
63 |
+
## 📥 Download Links:
|
64 |
+
|
65 |
+
- [abandoned_50_epochs.safetensors](./abandoned_50_epochs.safetensors) - LoRA Model File
|
66 |
+
- [wan_txt2vid_lora_workflow.json](./workflow/wan_txt2vid_lora_workflow.json) - Wan T2V with LoRA Workflow for ComfyUI
|
67 |
+
|
68 |
+
---
|
69 |
+
<div style="background-color: #f8f9fa; padding: 20px; border-radius: 10px; margin-bottom: 20px;">
|
70 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
71 |
+
<h2 style="color: #24292e; margin-top: 0;">Recommended Settings</h2>
|
72 |
+
<ul style="margin-bottom: 0;">
|
73 |
+
<li><b>LoRA Strength:</b> 1.0</li>
|
74 |
+
<li><b>Embedded Guidance Scale:</b> 6.0</li>
|
75 |
+
<li><b>Flow Shift:</b> 5.0</li>
|
76 |
+
</ul>
|
77 |
+
</div>
|
78 |
+
|
79 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
80 |
+
<h2 style="color: #24292e; margin-top: 0;">Trigger Words</h2>
|
81 |
+
<p>The key trigger phrase is: <code style="background-color: #f0f0f0; padding: 3px 6px; border-radius: 4px;">abandoned places</code></p>
|
82 |
+
</div>
|
83 |
+
|
84 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
85 |
+
<h2 style="color: #24292e; margin-top: 0;">Prompt Template</h2>
|
86 |
+
<p>For prompting, check out the example prompts; this way of prompting seems to work very well.</p>
|
87 |
+
|
88 |
+
|
89 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
90 |
+
<h2 style="color: #24292e; margin-top: 0;">ComfyUI Workflow</h2>
|
91 |
+
<p>This LoRA works with a modified version of <a href="https://github.com/kijai/ComfyUI-WanVideoWrapper/blob/main/example_workflows/wanvideo_T2V_example_02.json" style="color: #0366d6; text-decoration: none;">Kijai's Wan Video Wrapper workflow</a>. The main modification is adding a Wan LoRA node connected to the base model.</p>
|
92 |
+
<img src="./workflow/workflow_screenshot.png" style="width: 100%; border-radius: 8px; margin: 15px 0; box-shadow: 0 4px 8px rgba(0,0,0,0.1);">
|
93 |
+
<p>See the Downloads section above for the modified workflow.</p>
|
94 |
+
</div>
|
95 |
+
</div>
|
96 |
+
|
97 |
+
<div style="background-color: #f8f9fa; padding: 20px; border-radius: 10px; margin-bottom: 20px;">
|
98 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
99 |
+
<h2 style="color: #24292e; margin-top: 0;">Model Information</h2>
|
100 |
+
<p>The model weights are available in Safetensors format. See the Downloads section above.</p>
|
101 |
+
</div>
|
102 |
+
|
103 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
104 |
+
<h2 style="color: #24292e; margin-top: 0;">Training Details</h2>
|
105 |
+
<ul style="margin-bottom: 0;">
|
106 |
+
<li><b>Base Model:</b> Wan2.1 14B T2V</li>
|
107 |
+
<li><b>Training Data:</b> Trained on 6 minutes of video comprised of 98 short clips (each clip captioned separately) of various clips of abandoned places.</li>
|
108 |
+
<li><b> Epochs:</b> 50</li>
|
109 |
+
</ul>
|
110 |
+
</div>
|
111 |
+
|
112 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
113 |
+
<h2 style="color: #24292e; margin-top: 0;">Additional Information</h2>
|
114 |
+
<p>Training was done using <a href="https://github.com/tdrussell/diffusion-pipe" style="color: #0366d6; text-decoration: none;">Diffusion Pipe for Training</a></p>
|
115 |
+
</div>
|
116 |
+
|
117 |
+
<div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
|
118 |
+
<h2 style="color: #24292e; margin-top: 0;">Acknowledgments</h2>
|
119 |
+
<p style="margin-bottom: 0;">Special thanks to Kijai for the ComfyUI Wan Video Wrapper and tdrussell for the training scripts!</p>
|
120 |
+
</div>
|
121 |
+
</div>
|