title: Towards Embodied Cognition in Robots via Spatially Grounded Synthetic
    Worlds
summary: We present a conceptual framework for training Vision-Language Models (VLMs)
  to perform Visual Perspective Taking (VPT), a core capability for embodied
  cognition essential for Human-Robot Interaction (HRI). As a first step toward
  this goal, we introduce a synthetic dataset, generated in NVIDIA Omniverse,
  that enables supervised learning for spatial reasoning tasks. Each instance
  includes an RGB image, a natural language description, and a ground-truth 4X4
  transformation matrix representing object pose. We focus on inferring Z-axis
  distance as a foundational skill, with future extensions targeting full 6
  Degrees Of Freedom (DOFs) reasoning. The dataset is publicly available to
  support further research. This work serves as a foundational step toward
  embodied AI systems capable of spatial understanding in interactive human-robot
  scenarios.
paper_authors._id: 682dd1aaf3e4b74b546b6db4
paper_authors.name: Joel Currie
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-21T13:15:52.516Z
paper_authors._id: 6793cc6e09bb7d2e0441a335
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6793cc6e09bb7d2e0441a335/QtAEDNeEJFvJuqbvvkvj3.jpeg
paper_authors.fullname: Joel Currie
paper_authors.type: user
paper_authors.user: jwgcurrie
paper_authors._id: 682dd1aaf3e4b74b546b6db5
paper_authors.name: Gioele Migno
paper_authors._id: 682dd1aaf3e4b74b546b6db6
paper_authors.name: Enrico Piacenti
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-21T13:14:19.458Z
paper_authors._id: 67d9304651046c588116e552
paper_authors.avatarUrl: /avatars/796426f7a7e024fba594b0c56ece8a57.svg
paper_authors.fullname: Enrico Piacenti
paper_authors.type: user
paper_authors.user: ep10102
paper_authors._id: 682dd1aaf3e4b74b546b6db7
paper_authors.name: Maria Elena Giannaccini
paper_authors._id: 682dd1aaf3e4b74b546b6db8
paper_authors.name: Patric Bach
paper_authors._id: 682dd1aaf3e4b74b546b6db9
paper_authors.name: Davide De Tommaso
paper_authors._id: 682dd1aaf3e4b74b546b6dba
paper_authors.name: Agnieszka Wykowska
paper_title: Towards Embodied Cognition in Robots via Spatially Grounded Synthetic
    Worlds
paper_summary: We present a conceptual framework for training Vision-Language Models (VLMs)
  to perform Visual Perspective Taking (VPT), a core capability for embodied
  cognition essential for Human-Robot Interaction (HRI). As a first step toward
  this goal, we introduce a synthetic dataset, generated in NVIDIA Omniverse,
  that enables supervised learning for spatial reasoning tasks. Each instance
  includes an RGB image, a natural language description, and a ground-truth 4X4
  transformation matrix representing object pose. We focus on inferring Z-axis
  distance as a foundational skill, with future extensions targeting full 6
  Degrees Of Freedom (DOFs) reasoning. The dataset is publicly available to
  support further research. This work serves as a foundational step toward
  embodied AI systems capable of spatial understanding in interactive human-robot
  scenarios.
paper_ai_summary: A synthetic dataset in NVIDIA Omniverse aids in training Vision-Language Models for Visual Perspective Taking by providing supervised learning for spatial reasoning tasks.
paper_ai_keywords: Vision-Language Models Visual Perspective Taking embodied cognition Human-Robot Interaction NVIDIA Omniverse spatial reasoning RGB image natural language description 4X4 transformation matrix object pose Z-axis distance Degrees Of Freedom