title: FinePhys: Fine-grained Human Action Generation by Explicitly
    Incorporating Physical Laws for Effective Skeletal Guidance
summary: Despite significant advances in video generation, synthesizing physically
  plausible human actions remains a persistent challenge, particularly in
  modeling fine-grained semantics and complex temporal dynamics. For instance,
  generating gymnastics routines such as "switch leap with 0.5 turn" poses
  substantial difficulties for current methods, often yielding unsatisfactory
  results. To bridge this gap, we propose FinePhys, a Fine-grained human action
  generation framework that incorporates Physics to obtain effective skeletal
  guidance. Specifically, FinePhys first estimates 2D poses in an online manner
  and then performs 2D-to-3D dimension lifting via in-context learning. To
  mitigate the instability and limited interpretability of purely data-driven 3D
  poses, we further introduce a physics-based motion re-estimation module
  governed by Euler-Lagrange equations, calculating joint accelerations via
  bidirectional temporal updating. The physically predicted 3D poses are then
  fused with data-driven ones, offering multi-scale 2D heatmap guidance for the
  diffusion process. Evaluated on three fine-grained action subsets from FineGym
  (FX-JUMP, FX-TURN, and FX-SALTO), FinePhys significantly outperforms
  competitive baselines. Comprehensive qualitative results further demonstrate
  FinePhys's ability to generate more natural and plausible fine-grained human
  actions.
paper_authors._id: 682bfc257f2ade8dcbef284d
paper_authors.name: Dian Shao
paper_authors._id: 682bfc257f2ade8dcbef284e
paper_authors.name: Mingfei Shi
paper_authors._id: 682bfc257f2ade8dcbef284f
paper_authors.name: Shengda Xu
paper_authors._id: 682bfc257f2ade8dcbef2850
paper_authors.name: Haodong Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:20:24.383Z
paper_authors._id: 6570450a78d7aca0c361a177
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/z0GrnXEsjK2_G-hFfQhKv.jpeg
paper_authors.fullname: Harold Chen
paper_authors.type: user
paper_authors.user: Harold328
paper_authors._id: 682bfc257f2ade8dcbef2851
paper_authors.name: Yongle Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:44:22.708Z
paper_authors._id: 673e1ae7c90f9c7fbe4298d7
paper_authors.avatarUrl: /avatars/a6f0e64af7c502beb4c1d91ff4c4ea56.svg
paper_authors.fullname: Yongle Huang
paper_authors.type: user
paper_authors.user: Jason-Huang824
paper_authors._id: 682bfc257f2ade8dcbef2852
paper_authors.name: Binglu Wang
paper_title: FinePhys: Fine-grained Human Action Generation by Explicitly
    Incorporating Physical Laws for Effective Skeletal Guidance
paper_summary: Despite significant advances in video generation, synthesizing physically
  plausible human actions remains a persistent challenge, particularly in
  modeling fine-grained semantics and complex temporal dynamics. For instance,
  generating gymnastics routines such as "switch leap with 0.5 turn" poses
  substantial difficulties for current methods, often yielding unsatisfactory
  results. To bridge this gap, we propose FinePhys, a Fine-grained human action
  generation framework that incorporates Physics to obtain effective skeletal
  guidance. Specifically, FinePhys first estimates 2D poses in an online manner
  and then performs 2D-to-3D dimension lifting via in-context learning. To
  mitigate the instability and limited interpretability of purely data-driven 3D
  poses, we further introduce a physics-based motion re-estimation module
  governed by Euler-Lagrange equations, calculating joint accelerations via
  bidirectional temporal updating. The physically predicted 3D poses are then
  fused with data-driven ones, offering multi-scale 2D heatmap guidance for the
  diffusion process. Evaluated on three fine-grained action subsets from FineGym
  (FX-JUMP, FX-TURN, and FX-SALTO), FinePhys significantly outperforms
  competitive baselines. Comprehensive qualitative results further demonstrate
  FinePhys's ability to generate more natural and plausible fine-grained human
  actions.
paper_ai_summary: FinePhys is a framework that integrates physics principles into deep learning for generating realistic human actions by combining 2D-to-3D pose estimation with Euler-Lagrange-based motion re-estimation, significantly improving the quality of fine-grained action synthesis.
paper_ai_keywords: FinePhys 2D poses 2D-to-3D dimension lifting in-context learning physics-based motion re-estimation Euler-Lagrange equations bidirectional temporal updating diffusion process multi-scale 2D heatmap guidance FineGym FX-JUMP FX-TURN FX-SALTO