  Analyzing and understanding hand information from multimedia materials like
images or videos is important for many real world applications and remains
active in research community. There are various works focusing on recovering
hand information from single image, however, they usually solve a single task,
for example, hand mask segmentation, 2D/3D hand pose estimation, or hand mesh
reconstruction and perform not well in challenging scenarios. To further
improve the performance of these tasks, we propose a novel Hand Image
Understanding (HIU) framework to extract comprehensive information of the hand
object from a single RGB image, by jointly considering the relationships
between these tasks. To achieve this goal, a cascaded multi-task learning (MTL)
backbone is designed to estimate the 2D heat maps, to learn the segmentation
mask, and to generate the intermediate 3D information encoding, followed by a
coarse-to-fine learning paradigm and a self-supervised learning strategy.
Qualitative experiments demonstrate that our approach is capable of recovering
reasonable mesh representations even in challenging situations. Quantitatively,
our method significantly outperforms the state-of-the-art approaches on various
widely-used datasets, in terms of diverse evaluation metrics.

Computer Vision and Pattern Recognition