Video Editing via Factorized Diffusion Distillation
We introduce Emu Video Edit (EVE), a model that establishes a new
state-of-the art in video editing without relying on any supervised video
editing data. To develop EVE we separately train an image editing adapter and a
video generation adapter, and attach both to the same text-to-image model.
Then, to align the adapters towards video editing we introduce a new
unsupervised distillation procedure, Factorized Diffusion Distillation. This
procedure distills knowledge from one or more teachers simultaneously, without
any supervised data. We utilize this procedure to teach EVE to edit videos by
jointly distilling knowledge to (i) precisely edit each individual frame from
the image editing adapter, and (ii) ensure temporal consistency among the
edited frames using the video generation adapter. Finally, to demonstrate the
potential of our approach in unlocking other capabilities, we align additional
combinations of adapters
65f3b141d09b5f817d5d85a6
Uriel Singer
claimed_verified
2024-03-15T08:03:13.077Z
6345b71843f4f2d2ed113355
/avatars/a497669a4c53a724c4f6ea615d1dda59.svg
Uriel Singer
user
urielsinger
65f3b141d09b5f817d5d85a7
Amit Zohar
admin_assigned
2024-03-15T08:17:19.261Z
64d8cb34505306fcd2fb89c3
/avatars/4ae9ffe5d494a7d53bc519ff8e402b3e.svg
Amit Zohar
user
amitz
65f3b141d09b5f817d5d85a8
Yuval Kirstain
admin_assigned
2024-03-15T08:17:25.887Z
604f82d33050a33ebb17ef65
/avatars/0727b7175a077b28035732352f5be171.svg
Yuval Kirstain
user
yuvalkirstain
65f3b141d09b5f817d5d85a9
Shelly Sheynin
admin_assigned
2024-03-15T08:17:33.313Z
63468e6ca6f101c7f9049132
https://cdn-avatars.huggingface.co/v1/production/uploads/1665662111525-63468e6ca6f101c7f9049132.png
Shelly Sheynin
user
shellysheynin
65f3b141d09b5f817d5d85aa
Adam Polyak
admin_assigned
2024-03-15T08:17:40.192Z
6304d514dae2eb7d08413d62
/avatars/02a571bc791b78d3993d9a0484b70a29.svg
Adam Polyak
user
adampo
65f3b141d09b5f817d5d85ab
Devi Parikh
admin_assigned
2024-03-15T08:17:48.027Z
6340753995c20b9447379c67
/avatars/f793c00a98335cd8f7efce41d403da16.svg
Devi Parikh
user
deviparikh-genai
65f3b141d09b5f817d5d85ac
Yaniv Taigman
Video Editing via Factorized Diffusion Distillation
We introduce Emu Video Edit (EVE), a model that establishes a new
state-of-the art in video editing without relying on any supervised video
editing data. To develop EVE we separately train an image editing adapter and a
video generation adapter, and attach both to the same text-to-image model.
Then, to align the adapters towards video editing we introduce a new
unsupervised distillation procedure, Factorized Diffusion Distillation. This
procedure distills knowledge from one or more teachers simultaneously, without
any supervised data. We utilize this procedure to teach EVE to edit videos by
jointly distilling knowledge to (i) precisely edit each individual frame from
the image editing adapter, and (ii) ensure temporal consistency among the
edited frames using the video generation adapter. Finally, to demonstrate the
potential of our approach in unlocking other capabilities, we align additional
combinations of adapters
Emu Video Edit (EVE) uses unsupervised distillation, specifically Factorized Diffusion Distillation, to perform video editing by aligning image editing and video generation adapters without needing supervised data.
text-to-image model Factorized Diffusion Distillation