V-Express: Conditional Dropout for Progressive Training of Portrait
  Video Generation
In the field of portrait video generation, the use of single images to
generate portrait videos has become increasingly prevalent. A common approach
involves leveraging generative models to enhance adapters for controlled
generation. However, control signals (e.g., text, audio, reference image, pose,
depth map, etc.) can vary in strength. Among these, weaker conditions often
struggle to be effective due to interference from stronger conditions, posing a
challenge in balancing these conditions. In our work on portrait video
generation, we identified audio signals as particularly weak, often
overshadowed by stronger signals such as facial pose and reference image.
However, direct training with weak signals often leads to difficulties in
convergence. To address this, we propose V-Express, a simple method that
balances different control signals through the progressive training and the
conditional dropout operation. Our method gradually enables effective control
by weak conditions, thereby achieving generation capabilities that
simultaneously take into account the facial pose, reference image, and audio.
The experimental results demonstrate that our method can effectively generate
portrait videos controlled by audio. Furthermore, a potential solution is
provided for the simultaneous and effective use of conditions of varying
strengths.
665fe5fe3be5defb09f3d6fc
Cong Wang
665fe5fe3be5defb09f3d6fd
Kuan Tian
665fe5fe3be5defb09f3d6fe
Jun Zhang
665fe5fe3be5defb09f3d6ff
Yonghang Guan
665fe5fe3be5defb09f3d700
Feng Luo
665fe5fe3be5defb09f3d701
Fei Shen
claimed_verified
2025-07-28T08:48:22.500Z
64982b945551ac0d48d1b340
/avatars/cd1178d731d0375da05248a32f0e67b7.svg
Fei Shen
user
feishen29
665fe5fe3be5defb09f3d702
Zhiwei Jiang
admin_assigned
2024-06-05T10:43:26.914Z
66446066b5a21591719f3f8e
/avatars/923e7786da7bdaa69c7e3d1567a6a156.svg
Jiang Zhiwei
user
Laziobird
665fe5fe3be5defb09f3d703
Qing Gu
665fe5fe3be5defb09f3d704
Xiao Han
665fe5fe3be5defb09f3d705
Wei Yang
V-Express: Conditional Dropout for Progressive Training of Portrait
  Video Generation
In the field of portrait video generation, the use of single images to
generate portrait videos has become increasingly prevalent. A common approach
involves leveraging generative models to enhance adapters for controlled
generation. However, control signals (e.g., text, audio, reference image, pose,
depth map, etc.) can vary in strength. Among these, weaker conditions often
struggle to be effective due to interference from stronger conditions, posing a
challenge in balancing these conditions. In our work on portrait video
generation, we identified audio signals as particularly weak, often
overshadowed by stronger signals such as facial pose and reference image.
However, direct training with weak signals often leads to difficulties in
convergence. To address this, we propose V-Express, a simple method that
balances different control signals through the progressive training and the
conditional dropout operation. Our method gradually enables effective control
by weak conditions, thereby achieving generation capabilities that
simultaneously take into account the facial pose, reference image, and audio.
The experimental results demonstrate that our method can effectively generate
portrait videos controlled by audio. Furthermore, a potential solution is
provided for the simultaneous and effective use of conditions of varying
strengths.
V-Express addresses the challenge of weak control signals in portrait video generation through progressive training and conditional dropout, enabling effective audio-controlled video generation alongside other conditions.
V-Express progressive training conditional dropout portrait video generation control signals audio signals facial pose reference image