title: Inflation with Diffusion: Efficient Temporal Adaptation for
    Text-to-Video Super-Resolution
summary: We propose an efficient diffusion-based text-to-video super-resolution (SR)
  tuning approach that leverages the readily learned capacity of pixel level
  image diffusion model to capture spatial information for video generation. To
  accomplish this goal, we design an efficient architecture by inflating the
  weightings of the text-to-image SR model into our video generation framework.
  Additionally, we incorporate a temporal adapter to ensure temporal coherence
  across video frames. We investigate different tuning approaches based on our
  inflated architecture and report trade-offs between computational costs and
  super-resolution quality. Empirical evaluation, both quantitative and
  qualitative, on the Shutterstock video dataset, demonstrates that our approach
  is able to perform text-to-video SR generation with good visual quality and
  temporal consistency. To evaluate temporal coherence, we also present
  visualizations in video format in
  https://drive.google.com/drive/folders/1YVc-KMSJqOrEUdQWVaI-Yfu8Vsfu_1aO?usp=sharing .
paper_authors._id: 65adcf8a356bf23b4a954769
paper_authors.name: Xin Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-22T21:11:27.773Z
paper_authors._id: 65aeb94cda1c18bd6dbd3319
paper_authors.avatarUrl: /avatars/c01cd557875715861fab2a0d9b390b4d.svg
paper_authors.fullname: Xin Yuan
paper_authors.type: user
paper_authors.user: yuanxzzz
paper_authors._id: 65adcf8a356bf23b4a95476a
paper_authors.name: Jinoo Baek
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-24T08:09:37.994Z
paper_authors._id: 636623a9aa6a4af607370a47
paper_authors.avatarUrl: /avatars/ae2f377e7d85b22ec244f90c8434b87f.svg
paper_authors.fullname: Jinwoo Baek
paper_authors.type: user
paper_authors.user: jinoo
paper_authors._id: 65adcf8a356bf23b4a95476b
paper_authors.name: Keyang Xu
paper_authors._id: 65adcf8a356bf23b4a95476c
paper_authors.name: Omer Tov
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-22T14:51:45.262Z
paper_authors._id: 623889093657dc674c30954e
paper_authors.avatarUrl: /avatars/a5a193d231e01448cc78ed5bba52bff5.svg
paper_authors.fullname: Omer Tov
paper_authors.type: user
paper_authors.user: omertov
paper_authors._id: 65adcf8a356bf23b4a95476d
paper_authors.name: Hongliang Fei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-22T21:11:30.504Z
paper_authors._id: 65aeb55e66ad44e2d567620c
paper_authors.avatarUrl: /avatars/36b98e143c465b852a26f14307e571ca.svg
paper_authors.fullname: Hongliang Fei
paper_authors.type: user
paper_authors.user: feihongliang0
paper_title: Inflation with Diffusion: Efficient Temporal Adaptation for
    Text-to-Video Super-Resolution
paper_summary: We propose an efficient diffusion-based text-to-video super-resolution (SR)
  tuning approach that leverages the readily learned capacity of pixel level
  image diffusion model to capture spatial information for video generation. To
  accomplish this goal, we design an efficient architecture by inflating the
  weightings of the text-to-image SR model into our video generation framework.
  Additionally, we incorporate a temporal adapter to ensure temporal coherence
  across video frames. We investigate different tuning approaches based on our
  inflated architecture and report trade-offs between computational costs and
  super-resolution quality. Empirical evaluation, both quantitative and
  qualitative, on the Shutterstock video dataset, demonstrates that our approach
  is able to perform text-to-video SR generation with good visual quality and
  temporal consistency. To evaluate temporal coherence, we also present
  visualizations in video format in
  https://drive.google.com/drive/folders/1YVc-KMSJqOrEUdQWVaI-Yfu8Vsfu_1aO?usp=sharing .
paper_ai_keywords: diffusion-based text-to-video super-resolution image diffusion model inflating the weightings text-to-image SR model temporal adapter temporal coherence video frames computational costs super-resolution quality Shutterstock video dataset visual quality temporal consistency