TeslaYang123/TC-Light
Video-to-Video
•
Updated
•
100
•
3
Yu li
Yukkkop
AI & ML interests
None yet
Recent Activity
liked
a model
about 10 hours ago
TeslaYang123/TC-Light
upvoted
a
paper
about 10 hours ago
TC-Light: Temporally Consistent Relighting for Dynamic Long Videos
upvoted
a
paper
1 day ago
Neural-Driven Image Editing
Organizations
None yet
liked
a
model
about 10 hours ago
upvoted
a
paper
about 10 hours ago
upvoted
a
paper
1 day ago
reacted to
Kseniase's
post with 👍
2 days ago
Post
4899
13 New types of LoRA
LoRA (Low-Rank Adaptation) is a popular lightweight method for fine-tuning AI models. It doesn't update the full model, it adds small trainable components, low-rank matrices, while keeping the original weights frozen. Only these adapters are trained.
Recently, many interesting new LoRA variations came out, so it’s a great time to take a look at these 13 clever approaches:
1. T-LoRA → T-LoRA: Single Image Diffusion Model Customization Without Overfitting (2507.05964)
A timestep-dependent LoRA method for adapting diffusion models with a single image. It dynamically adjusts updates and uses orthogonal initialization to reduce overlap, achieving better fidelity–alignment balance than standard LoRA
2. SingLoRA → SingLoRA: Low Rank Adaptation Using a Single Matrix (2507.05566)
Simplifies LoRA by using only one small matrix instead of usual two, and multiplying it by its own transpose (like A × Aᵀ). It uses half the parameters of LoRA and avoids scale mismatch between different matrices
3. LiON-LoRA → LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion (2507.05678)
Improves control and precision in video diffusion models when training data is limited. It builds on LoRA, adding 3 key principles: linear scalability, orthogonality, and norm consistency. A controllable token and modified self-attention enables smooth adjustment of motion
4. LoRA-Mixer → LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing (2507.00029)
Combines LoRA and mixture-of-experts (MoE) to adapt LLMs for multiple tasks. It dynamically routes task-specific LoRA experts into linear projections of attention modules, supporting both joint training and frozen expert reuse
5. QR-LoRA → QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation (2507.04599)
Separates content and style when combining multiple LoRA adapters. It implements QR decomposition to structure parameter updates, where the orthogonal Q matrix reduces interference between features, and the R matrix captures specific transformations
Read further in the comments 👇
If you like it, also subscribe to the Turing Post: https://www.turingpost.com/subscribe
LoRA (Low-Rank Adaptation) is a popular lightweight method for fine-tuning AI models. It doesn't update the full model, it adds small trainable components, low-rank matrices, while keeping the original weights frozen. Only these adapters are trained.
Recently, many interesting new LoRA variations came out, so it’s a great time to take a look at these 13 clever approaches:
1. T-LoRA → T-LoRA: Single Image Diffusion Model Customization Without Overfitting (2507.05964)
A timestep-dependent LoRA method for adapting diffusion models with a single image. It dynamically adjusts updates and uses orthogonal initialization to reduce overlap, achieving better fidelity–alignment balance than standard LoRA
2. SingLoRA → SingLoRA: Low Rank Adaptation Using a Single Matrix (2507.05566)
Simplifies LoRA by using only one small matrix instead of usual two, and multiplying it by its own transpose (like A × Aᵀ). It uses half the parameters of LoRA and avoids scale mismatch between different matrices
3. LiON-LoRA → LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion (2507.05678)
Improves control and precision in video diffusion models when training data is limited. It builds on LoRA, adding 3 key principles: linear scalability, orthogonality, and norm consistency. A controllable token and modified self-attention enables smooth adjustment of motion
4. LoRA-Mixer → LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing (2507.00029)
Combines LoRA and mixture-of-experts (MoE) to adapt LLMs for multiple tasks. It dynamically routes task-specific LoRA experts into linear projections of attention modules, supporting both joint training and frozen expert reuse
5. QR-LoRA → QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation (2507.04599)
Separates content and style when combining multiple LoRA adapters. It implements QR decomposition to structure parameter updates, where the orthogonal Q matrix reduces interference between features, and the R matrix captures specific transformations
Read further in the comments 👇
If you like it, also subscribe to the Turing Post: https://www.turingpost.com/subscribe
- 1 reply
liked
2
models
4 days ago
liked
a
model
6 days ago
upvoted
a
paper
7 days ago
liked
a
model
7 days ago
liked
a
model
8 days ago
reacted to
nicolay-r's
post with 🔥
8 days ago
Post
3506
🚀 For those who interested in summarization of the long textual reports in medical domain 📝🩺,
@Xiaolihai
and I delighted to share that we experiment with distillation tuning adaptation for Qwen-2.5 0.5B. We use reports from the MultiClinSum dataset and pass it through 72B version to retrieve report explanations in order to initiate ditillation tuning for 0.5B model. We experiment with passages written in English, French, Portuguese, and Spanish.
🔑 We find that using distil-technique results in 2-4% performance increment on fine-tuning and similar improvements for reports in English (non-official and official evaluation). For the other it results in systems that perform similar to the convential tuning (standard) (see result below).
Dataset: https://zenodo.org/records/15459174
Competition: https://participants-area.bioasq.org/general_information/MultiClinSum/
Github: https://github.com/nicolay-r/distil-tuning-llm
model: nicolay-r/qwen25-05b-multiclinsum-distil
🔑 We find that using distil-technique results in 2-4% performance increment on fine-tuning and similar improvements for reports in English (non-official and official evaluation). For the other it results in systems that perform similar to the convential tuning (standard) (see result below).
Dataset: https://zenodo.org/records/15459174
Competition: https://participants-area.bioasq.org/general_information/MultiClinSum/
Github: https://github.com/nicolay-r/distil-tuning-llm
model: nicolay-r/qwen25-05b-multiclinsum-distil
- 3 replies
upvoted
a
paper
10 days ago
liked
a
model
13 days ago
liked
a
model
15 days ago
liked
a
model
17 days ago
liked
a
model
22 days ago
upvoted
a
paper
24 days ago
liked
a
model
24 days ago
upvoted
a
paper
24 days ago
liked
a
model
26 days ago
