title: Controllable Human-centric Keyframe Interpolation with Generative Prior
summary: Existing interpolation methods use pre-trained video diffusion priors to
  generate intermediate frames between sparsely sampled keyframes. In the absence
  of 3D geometric guidance, these methods struggle to produce plausible results
  for complex, articulated human motions and offer limited control over the
  synthesized dynamics. In this paper, we introduce PoseFuse3D Keyframe
  Interpolator (PoseFuse3D-KI), a novel framework that integrates 3D human
  guidance signals into the diffusion process for Controllable Human-centric
  Keyframe Interpolation (CHKI). To provide rich spatial and structural cues for
  interpolation, our PoseFuse3D, a 3D-informed control model, features a novel
  SMPL-X encoder that transforms 3D geometry and shape into the 2D latent
  conditioning space, alongside a fusion network that integrates these 3D cues
  with 2D pose embeddings. For evaluation, we build CHKI-Video, a new dataset
  annotated with both 2D poses and 3D SMPL-X parameters. We show that
  PoseFuse3D-KI consistently outperforms state-of-the-art baselines on
  CHKI-Video, achieving a 9% improvement in PSNR and a 38% reduction in LPIPS.
  Comprehensive ablations demonstrate that our PoseFuse3D model improves
  interpolation fidelity.
paper_authors._id: 68407ac4769b3772e8fdb16a
paper_authors.name: Zujin Guo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:27:36.035Z
paper_authors._id: 62f4b57a9437c4d04fb5e6d6
paper_authors.avatarUrl: /avatars/3681e4948fba1317c80bbbf2326a5fcd.svg
paper_authors.fullname: ZUJIN GUO
paper_authors.type: user
paper_authors.user: GSean
paper_authors._id: 68407ac4769b3772e8fdb16b
paper_authors.name: Size Wu
paper_authors._id: 68407ac4769b3772e8fdb16c
paper_authors.name: Zhongang Cai
paper_authors._id: 68407ac4769b3772e8fdb16d
paper_authors.name: Wei Li
paper_authors._id: 68407ac4769b3772e8fdb16e
paper_authors.name: Chen Change Loy
paper_title: Controllable Human-centric Keyframe Interpolation with Generative Prior
paper_summary: Existing interpolation methods use pre-trained video diffusion priors to
  generate intermediate frames between sparsely sampled keyframes. In the absence
  of 3D geometric guidance, these methods struggle to produce plausible results
  for complex, articulated human motions and offer limited control over the
  synthesized dynamics. In this paper, we introduce PoseFuse3D Keyframe
  Interpolator (PoseFuse3D-KI), a novel framework that integrates 3D human
  guidance signals into the diffusion process for Controllable Human-centric
  Keyframe Interpolation (CHKI). To provide rich spatial and structural cues for
  interpolation, our PoseFuse3D, a 3D-informed control model, features a novel
  SMPL-X encoder that transforms 3D geometry and shape into the 2D latent
  conditioning space, alongside a fusion network that integrates these 3D cues
  with 2D pose embeddings. For evaluation, we build CHKI-Video, a new dataset
  annotated with both 2D poses and 3D SMPL-X parameters. We show that
  PoseFuse3D-KI consistently outperforms state-of-the-art baselines on
  CHKI-Video, achieving a 9% improvement in PSNR and a 38% reduction in LPIPS.
  Comprehensive ablations demonstrate that our PoseFuse3D model improves
  interpolation fidelity.
paper_ai_summary: PoseFuse3D Keyframe Interpolator integrates 3D human guidance into diffusion models for more accurate and controllable video frame interpolation.
paper_ai_keywords: video diffusion priors interpolation 3D human guidance PoseFuse3D Keyframe Interpolator Controllable Human-centric Keyframe Interpolation CHKI SMPL-X encoder 3D-informed control model fusion network 2D pose embeddings CHKI-Video PSNR LPIPS