Temporal Residual Jacobians For Rig-free Motion Transfer
We introduce Temporal Residual Jacobians as a novel representation to enable
data-driven motion transfer. Our approach does not assume access to any rigging
or intermediate shape keyframes, produces geometrically and temporally
consistent motions, and can be used to transfer long motion sequences. Central
to our approach are two coupled neural networks that individually predict local
geometric and temporal changes that are subsequently integrated, spatially and
temporally, to produce the final animated meshes. The two networks are jointly
trained, complement each other in producing spatial and temporal signals, and
are supervised directly with 3D positional information. During inference, in
the absence of keyframes, our method essentially solves a motion extrapolation
problem. We test our setup on diverse meshes (synthetic and scanned shapes) to
demonstrate its superiority in generating realistic and natural-looking
animations on unseen body shapes against SoTA alternatives. Supplemental video
and code are available at https://temporaljacobians.github.io/ .
669f1dce384dc4fd761642c0
Sanjeev Muralikrishnan
claimed_verified
2024-07-23T10:05:00.131Z
669f7e451f5eece012af95a2
/avatars/b6a148a0324041ca20efca580cec1e4d.svg
Sanjeev Muralikrishnan
user
sanjeevmk
669f1dce384dc4fd761642c1
Niladri Shekhar Dutt
claimed_verified
2024-07-23T10:05:02.122Z
65b3e258d5bf0d622a89701b
https://cdn-avatars.huggingface.co/v1/production/uploads/65b3e258d5bf0d622a89701b/Bv2Fv8Hyc6w5tvNCPTg2_.jpeg
Niladri Shekhar Dutt
user
niladridutt
669f1dce384dc4fd761642c2
Siddhartha Chaudhuri
669f1dce384dc4fd761642c3
Noam Aigerman
claimed_verified
2024-07-23T10:05:03.960Z
637ed06c1dbae09191065d0c
/avatars/93bb8b51ca27c89b3ddce1be08f641b3.svg
noam
user
noamaig
669f1dce384dc4fd761642c4
Vladimir Kim
669f1dce384dc4fd761642c5
Matthew Fisher
669f1dce384dc4fd761642c6
Niloy J. Mitra
Temporal Residual Jacobians For Rig-free Motion Transfer
We introduce Temporal Residual Jacobians as a novel representation to enable
data-driven motion transfer. Our approach does not assume access to any rigging
or intermediate shape keyframes, produces geometrically and temporally
consistent motions, and can be used to transfer long motion sequences. Central
to our approach are two coupled neural networks that individually predict local
geometric and temporal changes that are subsequently integrated, spatially and
temporally, to produce the final animated meshes. The two networks are jointly
trained, complement each other in producing spatial and temporal signals, and
are supervised directly with 3D positional information. During inference, in
the absence of keyframes, our method essentially solves a motion extrapolation
problem. We test our setup on diverse meshes (synthetic and scanned shapes) to
demonstrate its superiority in generating realistic and natural-looking
animations on unseen body shapes against SoTA alternatives. Supplemental video
and code are available at https://temporaljacobians.github.io/ .
Temporal Residual Jacobians enable data-driven motion transfer without keyframes by integrating local geometric and temporal changes into realistic animations.
Temporal Residual Jacobians neural networks 3D positional information motion extrapolation realistic animations