Learning Dense Hand Contact Estimation from Imbalanced Data
Hands are essential to human interaction, and understanding contact between
hands and the world can promote comprehensive understanding of their function.
Recently, there have been growing number of hand interaction datasets that
cover interaction with object, other hand, scene, and body. Despite the
significance of the task and increasing high-quality data, how to effectively
learn dense hand contact estimation remains largely underexplored. There are
two major challenges for learning dense hand contact estimation. First, there
exists class imbalance issue from hand contact datasets where majority of
samples are not in contact. Second, hand contact datasets contain spatial
imbalance issue with most of hand contact exhibited in finger tips, resulting
in challenges for generalization towards contacts in other hand regions. To
tackle these issues, we present a framework that learns dense HAnd COntact
estimation (HACO) from imbalanced data. To resolve the class imbalance issue,
we introduce balanced contact sampling, which builds and samples from multiple
sampling groups that fairly represent diverse contact statistics for both
contact and non-contact samples. Moreover, to address the spatial imbalance
issue, we propose vertex-level class-balanced (VCB) loss, which incorporates
spatially varying contact distribution by separately reweighting loss
contribution of each vertex based on its contact frequency across dataset. As a
result, we effectively learn to predict dense hand contact estimation with
large-scale hand contact data without suffering from class and spatial
imbalance issue. The codes will be released.
682a9a3f5e6f0c59f4d8a0e5
Daniel Sungho Jung
claimed_verified
2025-05-19T06:42:43.502Z
65601c6ee23401f82005e361
/avatars/e9fc24bd8c5afd8b07a2f42765d44a7d.svg
Daniel Sungho Jung
user
dqj5182
682a9a3f5e6f0c59f4d8a0e6
Kyoung Mu Lee
admin_assigned
2025-05-19T07:25:48.640Z
656056b21392aa3beb5de0bd
/avatars/07f25b750ef308d65f2e6c82506e7816.svg
Kyoung Mu  Lee 
user
kyoungmu
Learning Dense Hand Contact Estimation from Imbalanced Data
Hands are essential to human interaction, and understanding contact between
hands and the world can promote comprehensive understanding of their function.
Recently, there have been growing number of hand interaction datasets that
cover interaction with object, other hand, scene, and body. Despite the
significance of the task and increasing high-quality data, how to effectively
learn dense hand contact estimation remains largely underexplored. There are
two major challenges for learning dense hand contact estimation. First, there
exists class imbalance issue from hand contact datasets where majority of
samples are not in contact. Second, hand contact datasets contain spatial
imbalance issue with most of hand contact exhibited in finger tips, resulting
in challenges for generalization towards contacts in other hand regions. To
tackle these issues, we present a framework that learns dense HAnd COntact
estimation (HACO) from imbalanced data. To resolve the class imbalance issue,
we introduce balanced contact sampling, which builds and samples from multiple
sampling groups that fairly represent diverse contact statistics for both
contact and non-contact samples. Moreover, to address the spatial imbalance
issue, we propose vertex-level class-balanced (VCB) loss, which incorporates
spatially varying contact distribution by separately reweighting loss
contribution of each vertex based on its contact frequency across dataset. As a
result, we effectively learn to predict dense hand contact estimation with
large-scale hand contact data without suffering from class and spatial
imbalance issue. The codes will be released.
A framework addresses dense hand contact estimation from imbalanced datasets using balanced contact sampling and a vertex-level class-balanced loss to handle class and spatial imbalance issues.
dense hand contact estimation HACO balanced contact sampling VCB loss vertex-level class-balanced loss