title: GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment
    Generalization
summary: This paper introduces GET-Zero, a model architecture and training procedure
  for learning an embodiment-aware control policy that can immediately adapt to
  new hardware changes without retraining. To do so, we present Graph Embodiment
  Transformer (GET), a transformer model that leverages the embodiment graph
  connectivity as a learned structural bias in the attention mechanism. We use
  behavior cloning to distill demonstration data from embodiment-specific expert
  policies into an embodiment-aware GET model that conditions on the hardware
  configuration of the robot to make control decisions. We conduct a case study
  on a dexterous in-hand object rotation task using different configurations of a
  four-fingered robot hand with joints removed and with link length extensions.
  Using the GET model along with a self-modeling loss enables GET-Zero to
  zero-shot generalize to unseen variation in graph structure and link length,
  yielding a 20% improvement over baseline methods. All code and qualitative
  video results are on https://get-zero-paper.github.io
paper_authors._id: 669f206b352220a858f53375
paper_authors.name: Austin Patel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-23T12:47:31.986Z
paper_authors._id: 65dea524e87db33835824fc2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65dea524e87db33835824fc2/8VY4CFwof0kCTpKlPAW3E.jpeg
paper_authors.fullname: Austin Patel
paper_authors.type: user
paper_authors.user: austinpatel
paper_authors._id: 669f206b352220a858f53376
paper_authors.name: Shuran Song
paper_title: GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment
    Generalization
paper_summary: This paper introduces GET-Zero, a model architecture and training procedure
  for learning an embodiment-aware control policy that can immediately adapt to
  new hardware changes without retraining. To do so, we present Graph Embodiment
  Transformer (GET), a transformer model that leverages the embodiment graph
  connectivity as a learned structural bias in the attention mechanism. We use
  behavior cloning to distill demonstration data from embodiment-specific expert
  policies into an embodiment-aware GET model that conditions on the hardware
  configuration of the robot to make control decisions. We conduct a case study
  on a dexterous in-hand object rotation task using different configurations of a
  four-fingered robot hand with joints removed and with link length extensions.
  Using the GET model along with a self-modeling loss enables GET-Zero to
  zero-shot generalize to unseen variation in graph structure and link length,
  yielding a 20% improvement over baseline methods. All code and qualitative
  video results are on https://get-zero-paper.github.io
paper_ai_keywords: Graph Embodiment Transformer (GET) embodiment graph connectivity attention mechanism behavior cloning embodiment-specific expert policies hardware configuration dexterous in-hand object rotation task self-modeling loss zero-shot generalize graph structure link length