Temporal In-Context Fine-Tuning for Versatile Control of Video Diffusion
  Models
Recent advances in text-to-video diffusion models have enabled high-quality
video synthesis, but controllable generation remains challenging, particularly
under limited data and compute. Existing fine-tuning methods for conditional
generation often rely on external encoders or architectural modifications,
which demand large datasets and are typically restricted to spatially aligned
conditioning, limiting flexibility and scalability. In this work, we introduce
Temporal In-Context Fine-Tuning (TIC-FT), an efficient and versatile approach
for adapting pretrained video diffusion models to diverse conditional
generation tasks. Our key idea is to concatenate condition and target frames
along the temporal axis and insert intermediate buffer frames with
progressively increasing noise levels. These buffer frames enable smooth
transitions, aligning the fine-tuning process with the pretrained model's
temporal dynamics. TIC-FT requires no architectural changes and achieves strong
performance with as few as 10-30 training samples. We validate our method
across a range of tasks, including image-to-video and video-to-video
generation, using large-scale base models such as CogVideoX-5B and Wan-14B.
Extensive experiments show that TIC-FT outperforms existing baselines in both
condition fidelity and visual quality, while remaining highly efficient in both
training and inference. For additional results, visit
https://kinam0252.github.io/TIC-FT/
683e7cbf402acb186580d5ec
Kinam Kim
admin_assigned
2025-06-03T12:32:41.547Z
64797735a68454566356b708
/avatars/3424d022dd8ad29b56eb41814c5c3dee.svg
Kinam Kim
user
kinam0252
683e7cbf402acb186580d5ed
Junha Hyung
admin_assigned
2025-06-03T12:32:50.417Z
6331ed964db0a767bbee4706
/avatars/2ef3fa6083ca5208f4b87c91345267d9.svg
Junha Hyung
user
JunhaH
683e7cbf402acb186580d5ee
Jaegul Choo
admin_assigned
2025-06-03T12:33:03.788Z
64be3127805e5b64572da65c
/avatars/edd3e94ba9e375827cc75b164602bcac.svg
Jaegul Choo
user
joyfull78
Temporal In-Context Fine-Tuning for Versatile Control of Video Diffusion
  Models
Recent advances in text-to-video diffusion models have enabled high-quality
video synthesis, but controllable generation remains challenging, particularly
under limited data and compute. Existing fine-tuning methods for conditional
generation often rely on external encoders or architectural modifications,
which demand large datasets and are typically restricted to spatially aligned
conditioning, limiting flexibility and scalability. In this work, we introduce
Temporal In-Context Fine-Tuning (TIC-FT), an efficient and versatile approach
for adapting pretrained video diffusion models to diverse conditional
generation tasks. Our key idea is to concatenate condition and target frames
along the temporal axis and insert intermediate buffer frames with
progressively increasing noise levels. These buffer frames enable smooth
transitions, aligning the fine-tuning process with the pretrained model's
temporal dynamics. TIC-FT requires no architectural changes and achieves strong
performance with as few as 10-30 training samples. We validate our method
across a range of tasks, including image-to-video and video-to-video
generation, using large-scale base models such as CogVideoX-5B and Wan-14B.
Extensive experiments show that TIC-FT outperforms existing baselines in both
condition fidelity and visual quality, while remaining highly efficient in both
training and inference. For additional results, visit
https://kinam0252.github.io/TIC-FT/
Temporal In-Context Fine-Tuning (TIC-FT) enhances pretrained video diffusion models for diverse conditional generation tasks with minimal data and without architectural changes.
text-to-video diffusion models fine-tuning external encoders architectural modifications Temporal In-Context Fine-Tuning condition and target frames buffer frames noise levels smooth transitions pretrained video diffusion models image-to-video generation video-to-video generation CogVideoX-5B Wan-14B