GeneOH Diffusion: Towards Generalizable Hand-Object Interaction
  Denoising via Denoising Diffusion
In this work, we tackle the challenging problem of denoising hand-object
interactions (HOI). Given an erroneous interaction sequence, the objective is
to refine the incorrect hand trajectory to remove interaction artifacts for a
perceptually realistic sequence. This challenge involves intricate interaction
noise, including unnatural hand poses and incorrect hand-object relations,
alongside the necessity for robust generalization to new interactions and
diverse noise patterns. We tackle those challenges through a novel approach,
GeneOH Diffusion, incorporating two key designs: an innovative contact-centric
HOI representation named GeneOH and a new domain-generalizable denoising
scheme. The contact-centric representation GeneOH informatively parameterizes
the HOI process, facilitating enhanced generalization across various HOI
scenarios. The new denoising scheme consists of a canonical denoising model
trained to project noisy data samples from a whitened noise space to a clean
data manifold and a "denoising via diffusion" strategy which can handle input
trajectories with various noise patterns by first diffusing them to align with
the whitened noise space and cleaning via the canonical denoiser. Extensive
experiments on four benchmarks with significant domain variations demonstrate
the superior effectiveness of our method. GeneOH Diffusion also shows promise
for various downstream applications. Project website:
https://meowuu7.github.io/GeneOH-Diffusion/.
65d82658bc9081f6c904e035
Xueyi Liu
admin_assigned
2024-02-23T13:08:50.028Z
65b8070ad49f4330ab0ca5f7
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/t4fI-3djMfgXCchU_xpjL.png
Xueyi Liu
user
xymeow7
65d82658bc9081f6c904e036
Li Yi
GeneOH Diffusion: Towards Generalizable Hand-Object Interaction
  Denoising via Denoising Diffusion
In this work, we tackle the challenging problem of denoising hand-object
interactions (HOI). Given an erroneous interaction sequence, the objective is
to refine the incorrect hand trajectory to remove interaction artifacts for a
perceptually realistic sequence. This challenge involves intricate interaction
noise, including unnatural hand poses and incorrect hand-object relations,
alongside the necessity for robust generalization to new interactions and
diverse noise patterns. We tackle those challenges through a novel approach,
GeneOH Diffusion, incorporating two key designs: an innovative contact-centric
HOI representation named GeneOH and a new domain-generalizable denoising
scheme. The contact-centric representation GeneOH informatively parameterizes
the HOI process, facilitating enhanced generalization across various HOI
scenarios. The new denoising scheme consists of a canonical denoising model
trained to project noisy data samples from a whitened noise space to a clean
data manifold and a "denoising via diffusion" strategy which can handle input
trajectories with various noise patterns by first diffusing them to align with
the whitened noise space and cleaning via the canonical denoiser. Extensive
experiments on four benchmarks with significant domain variations demonstrate
the superior effectiveness of our method. GeneOH Diffusion also shows promise
for various downstream applications. Project website:
https://meowuu7.github.io/GeneOH-Diffusion/.
GeneOH Diffusion denoises hand-object interactions using a contact-centric representation and a denoising via diffusion strategy, achieving high effectiveness across various benchmarks.
denoising hand-object interactions hand trajectory interaction artifacts contact-centric HOI representation denoising scheme canonical denoising model denoising via diffusion whitened noise space clean data manifold