title: LTX-Video: Realtime Video Latent Diffusion
summary: We introduce LTX-Video, a transformer-based latent diffusion model that
  adopts a holistic approach to video generation by seamlessly integrating the
  responsibilities of the Video-VAE and the denoising transformer. Unlike
  existing methods, which treat these components as independent, LTX-Video aims
  to optimize their interaction for improved efficiency and quality. At its core
  is a carefully designed Video-VAE that achieves a high compression ratio of
  1:192, with spatiotemporal downscaling of 32 x 32 x 8 pixels per token, enabled
  by relocating the patchifying operation from the transformer's input to the
  VAE's input. Operating in this highly compressed latent space enables the
  transformer to efficiently perform full spatiotemporal self-attention, which is
  essential for generating high-resolution videos with temporal consistency.
  However, the high compression inherently limits the representation of fine
  details. To address this, our VAE decoder is tasked with both latent-to-pixel
  conversion and the final denoising step, producing the clean result directly in
  pixel space. This approach preserves the ability to generate fine details
  without incurring the runtime cost of a separate upsampling module. Our model
  supports diverse use cases, including text-to-video and image-to-video
  generation, with both capabilities trained simultaneously. It achieves
  faster-than-real-time generation, producing 5 seconds of 24 fps video at
  768x512 resolution in just 2 seconds on an Nvidia H100 GPU, outperforming all
  existing models of similar scale. The source code and pre-trained models are
  publicly available, setting a new benchmark for accessible and scalable video
  generation.
paper_authors._id: 6777ef01cbb36f508a37aa44
paper_authors.name: Yoav HaCohen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-05T23:02:34.545Z
paper_authors._id: 6303cc5e0547362a22a51af0
paper_authors.avatarUrl: /avatars/8f3348f121565bf6c5e1af0e559a43a3.svg
paper_authors.fullname: Yoav HaCohen
paper_authors.type: user
paper_authors.user: yoavhacohen
paper_authors._id: 6777ef01cbb36f508a37aa45
paper_authors.name: Nisan Chiprut
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:08:26.318Z
paper_authors._id: 62dd30a8d43078cd49ac8ad8
paper_authors.avatarUrl: /avatars/ad599719290637f7817b7508a91c2e2c.svg
paper_authors.fullname: Nisan Chiprut
paper_authors.type: user
paper_authors.user: nisan
paper_authors._id: 6777ef01cbb36f508a37aa46
paper_authors.name: Benny Brazowski
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-05T23:02:32.419Z
paper_authors._id: 6489c487b9e9258ba065418f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6489c487b9e9258ba065418f/6rzmV3bQ3YxswG6NP2hDW.png
paper_authors.fullname: Benny Brazowski
paper_authors.type: user
paper_authors.user: benibraz
paper_authors._id: 6777ef01cbb36f508a37aa47
paper_authors.name: Daniel Shalem
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:08:45.579Z
paper_authors._id: 672c8f988e6f8ed5741f4917
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/LPjQ1suwUAfipCIe1HbaB.png
paper_authors.fullname: daniel shalem
paper_authors.type: user
paper_authors.user: danielshalemltx
paper_authors._id: 6777ef01cbb36f508a37aa48
paper_authors.name: Dudu Moshe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:08:51.844Z
paper_authors._id: 636b97a57631fe5e86fe1fa2
paper_authors.avatarUrl: /avatars/c568ae26fd4fc2655cd12f15d539db58.svg
paper_authors.fullname: Dudu Moshe
paper_authors.type: user
paper_authors.user: dudumoshe
paper_authors._id: 6777ef01cbb36f508a37aa49
paper_authors.name: Eitan Richardson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:08:57.988Z
paper_authors._id: 677a422979d3c32a5dd87a0a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/WUa6E68GpnT2mEMJ41nDd.png
paper_authors.fullname: Eitan Richardson
paper_authors.type: user
paper_authors.user: eitanrich
paper_authors._id: 6777ef01cbb36f508a37aa4a
paper_authors.name: Eran Levin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:15:13.163Z
paper_authors._id: 670d24e76ad961f79c00fe9b
paper_authors.avatarUrl: /avatars/7407fa476f1b227e35817578632e839f.svg
paper_authors.fullname: Eran Levin
paper_authors.type: user
paper_authors.user: eranlevinlt
paper_authors._id: 6777ef01cbb36f508a37aa4b
paper_authors.name: Guy Shiran
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:15:21.216Z
paper_authors._id: 673f6911d83832a6ce15e7bf
paper_authors.avatarUrl: /avatars/0da6cded3b0e785241a6ba5fdb5d8ceb.svg
paper_authors.fullname: Guy Shiran
paper_authors.type: user
paper_authors.user: guysrn
paper_authors._id: 6777ef01cbb36f508a37aa4c
paper_authors.name: Nir Zabari
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:15:47.837Z
paper_authors._id: 673f29b568595672b8d3e90e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/673f29b568595672b8d3e90e/4sYADg3mpqMKmJ4fQwaTl.png
paper_authors.fullname: Nir Zabari
paper_authors.type: user
paper_authors.user: NirZabariLTX
paper_authors._id: 6777ef01cbb36f508a37aa4d
paper_authors.name: Ori Gordon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-05T23:02:14.348Z
paper_authors._id: 674348b46215a2c0878e219b
paper_authors.avatarUrl: /avatars/8a213e431a1583d1a93377410907c059.svg
paper_authors.fullname: Ori Gordon
paper_authors.type: user
paper_authors.user: origordon
paper_authors._id: 6777ef01cbb36f508a37aa4e
paper_authors.name: Poriya Panet
paper_authors._id: 6777ef01cbb36f508a37aa4f
paper_authors.name: Sapir Weissbuch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:16:36.788Z
paper_authors._id: 5fd0b693a23a118c84d66a2d
paper_authors.avatarUrl: /avatars/a469a67da5765b557444a40264dbe73f.svg
paper_authors.fullname: Sapir Weissbuch
paper_authors.type: user
paper_authors.user: sapirw
paper_authors._id: 6777ef01cbb36f508a37aa50
paper_authors.name: Victor Kulikov
paper_authors._id: 6777ef01cbb36f508a37aa51
paper_authors.name: Yaki Bitterman
paper_authors._id: 6777ef01cbb36f508a37aa52
paper_authors.name: Zeev Melumian
paper_authors._id: 6777ef01cbb36f508a37aa53
paper_authors.name: Ofir Bibi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-06T14:16:55.345Z
paper_authors._id: 631f58935ba8c026340b377c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631f58935ba8c026340b377c/4yoHLdNE99VBb7ji_Mzzj.jpeg
paper_authors.fullname: Ofir Bibi
paper_authors.type: user
paper_authors.user: ofirbibi
paper_title: LTX-Video: Realtime Video Latent Diffusion
paper_summary: We introduce LTX-Video, a transformer-based latent diffusion model that
  adopts a holistic approach to video generation by seamlessly integrating the
  responsibilities of the Video-VAE and the denoising transformer. Unlike
  existing methods, which treat these components as independent, LTX-Video aims
  to optimize their interaction for improved efficiency and quality. At its core
  is a carefully designed Video-VAE that achieves a high compression ratio of
  1:192, with spatiotemporal downscaling of 32 x 32 x 8 pixels per token, enabled
  by relocating the patchifying operation from the transformer's input to the
  VAE's input. Operating in this highly compressed latent space enables the
  transformer to efficiently perform full spatiotemporal self-attention, which is
  essential for generating high-resolution videos with temporal consistency.
  However, the high compression inherently limits the representation of fine
  details. To address this, our VAE decoder is tasked with both latent-to-pixel
  conversion and the final denoising step, producing the clean result directly in
  pixel space. This approach preserves the ability to generate fine details
  without incurring the runtime cost of a separate upsampling module. Our model
  supports diverse use cases, including text-to-video and image-to-video
  generation, with both capabilities trained simultaneously. It achieves
  faster-than-real-time generation, producing 5 seconds of 24 fps video at
  768x512 resolution in just 2 seconds on an Nvidia H100 GPU, outperforming all
  existing models of similar scale. The source code and pre-trained models are
  publicly available, setting a new benchmark for accessible and scalable video
  generation.
paper_ai_keywords: transformer-based latent diffusion model Video-VAE denoising transformer spatiotemporal downscaling patchifying operation latent space full spatiotemporal self-attention temporal consistency latent-to-pixel conversion finetuning text-to-video generation image-to-video generation