title: PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping
    with VLM
summary: This paper introduces PoseLess, a novel framework for robot hand control that
  eliminates the need for explicit pose estimation by directly mapping 2D images
  to joint angles using projected representations. Our approach leverages
  synthetic training data generated through randomized joint configurations,
  enabling zero-shot generalization to real-world scenarios and cross-morphology
  transfer from robotic to human hands. By projecting visual inputs and employing
  a transformer-based decoder, PoseLess achieves robust, low-latency control
  while addressing challenges such as depth ambiguity and data scarcity.
  Experimental results demonstrate competitive performance in joint angle
  prediction accuracy without relying on any human-labelled dataset.
paper_authors._id: 67d4d78897767f49253a0739
paper_authors.name: Alan Dao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-24T16:39:52.798Z
paper_authors._id: 62d7b2339b629105a5d6888a
paper_authors.avatarUrl: /avatars/c3f164fde6b8f9a671890e08ce8a3e75.svg
paper_authors.fullname: Alan Dao
paper_authors.type: user
paper_authors.user: alandao
paper_authors._id: 67d4d78897767f49253a073a
paper_authors.name: Dinh Bach Vu
paper_authors._id: 67d4d78897767f49253a073b
paper_authors.name: Tuan Le Duc Anh
paper_authors._id: 67d4d78897767f49253a073c
paper_authors.name: Bui Quang Huy
paper_title: PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping
    with VLM
paper_summary: This paper introduces PoseLess, a novel framework for robot hand control that
  eliminates the need for explicit pose estimation by directly mapping 2D images
  to joint angles using projected representations. Our approach leverages
  synthetic training data generated through randomized joint configurations,
  enabling zero-shot generalization to real-world scenarios and cross-morphology
  transfer from robotic to human hands. By projecting visual inputs and employing
  a transformer-based decoder, PoseLess achieves robust, low-latency control
  while addressing challenges such as depth ambiguity and data scarcity.
  Experimental results demonstrate competitive performance in joint angle
  prediction accuracy without relying on any human-labelled dataset.
paper_ai_summary: PoseLess uses synthetic training data and transformer-based decoders to map 2D images to joint angles for robot hand control, achieving zero-shot generalization to real-world scenarios and cross-morphology transfer.
paper_ai_keywords: transformer-based decoder