FinePhys: Fine-grained Human Action Generation by Explicitly
  Incorporating Physical Laws for Effective Skeletal Guidance
Despite significant advances in video generation, synthesizing physically
plausible human actions remains a persistent challenge, particularly in
modeling fine-grained semantics and complex temporal dynamics. For instance,
generating gymnastics routines such as "switch leap with 0.5 turn" poses
substantial difficulties for current methods, often yielding unsatisfactory
results. To bridge this gap, we propose FinePhys, a Fine-grained human action
generation framework that incorporates Physics to obtain effective skeletal
guidance. Specifically, FinePhys first estimates 2D poses in an online manner
and then performs 2D-to-3D dimension lifting via in-context learning. To
mitigate the instability and limited interpretability of purely data-driven 3D
poses, we further introduce a physics-based motion re-estimation module
governed by Euler-Lagrange equations, calculating joint accelerations via
bidirectional temporal updating. The physically predicted 3D poses are then
fused with data-driven ones, offering multi-scale 2D heatmap guidance for the
diffusion process. Evaluated on three fine-grained action subsets from FineGym
(FX-JUMP, FX-TURN, and FX-SALTO), FinePhys significantly outperforms
competitive baselines. Comprehensive qualitative results further demonstrate
FinePhys's ability to generate more natural and plausible fine-grained human
actions.
682bfc257f2ade8dcbef284d
Dian Shao
682bfc257f2ade8dcbef284e
Mingfei Shi
682bfc257f2ade8dcbef284f
Shengda Xu
682bfc257f2ade8dcbef2850
Haodong Chen
claimed_verified
2025-05-20T07:20:24.383Z
6570450a78d7aca0c361a177
https://cdn-avatars.huggingface.co/v1/production/uploads/6570450a78d7aca0c361a177/z0GrnXEsjK2_G-hFfQhKv.jpeg
Harold Chen
user
Harold328
682bfc257f2ade8dcbef2851
Yongle Huang
admin_assigned
2025-05-20T08:44:22.708Z
673e1ae7c90f9c7fbe4298d7
/avatars/a6f0e64af7c502beb4c1d91ff4c4ea56.svg
Yongle Huang
user
Jason-Huang824
682bfc257f2ade8dcbef2852
Binglu Wang
FinePhys: Fine-grained Human Action Generation by Explicitly
  Incorporating Physical Laws for Effective Skeletal Guidance
Despite significant advances in video generation, synthesizing physically
plausible human actions remains a persistent challenge, particularly in
modeling fine-grained semantics and complex temporal dynamics. For instance,
generating gymnastics routines such as "switch leap with 0.5 turn" poses
substantial difficulties for current methods, often yielding unsatisfactory
results. To bridge this gap, we propose FinePhys, a Fine-grained human action
generation framework that incorporates Physics to obtain effective skeletal
guidance. Specifically, FinePhys first estimates 2D poses in an online manner
and then performs 2D-to-3D dimension lifting via in-context learning. To
mitigate the instability and limited interpretability of purely data-driven 3D
poses, we further introduce a physics-based motion re-estimation module
governed by Euler-Lagrange equations, calculating joint accelerations via
bidirectional temporal updating. The physically predicted 3D poses are then
fused with data-driven ones, offering multi-scale 2D heatmap guidance for the
diffusion process. Evaluated on three fine-grained action subsets from FineGym
(FX-JUMP, FX-TURN, and FX-SALTO), FinePhys significantly outperforms
competitive baselines. Comprehensive qualitative results further demonstrate
FinePhys's ability to generate more natural and plausible fine-grained human
actions.
FinePhys is a framework that integrates physics principles into deep learning for generating realistic human actions by combining 2D-to-3D pose estimation with Euler-Lagrange-based motion re-estimation, significantly improving the quality of fine-grained action synthesis.
FinePhys 2D poses 2D-to-3D dimension lifting in-context learning physics-based motion re-estimation Euler-Lagrange equations bidirectional temporal updating diffusion process multi-scale 2D heatmap guidance FineGym FX-JUMP FX-TURN FX-SALTO