GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment Generalization
<|reference_start|>GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment Generalization: This paper introduces GET-Zero, a model architecture and training procedure for learning an embodiment-aware control policy that can immediately adapt to new hardware changes without retraining. To do so, we present Graph Embodiment Transformer (GET), a transformer model that leverages the embodiment graph connectivity as a learned structural bias in the attention mechanism. We use behavior cloning to distill demonstration data from embodiment-specific expert policies into an embodiment-aware GET model that conditions on the hardware configuration of the robot to make control decisions. We conduct a case study on a dexterous in-hand object rotation task using different configurations of a four-fingered robot hand with joints removed and with link length extensions. Using the GET model along with a self-modeling loss enables GET-Zero to zero-shot generalize to unseen variation in graph structure and link length, yielding a 20% improvement over baseline methods. All code and qualitative video results are on https://get-zero-paper.github.io<|reference_end|>
@article{patel2024get-zero:,
    title={GET-Zero: Graph Embodiment Transformer for Zero-shot Embodiment
  Generalization},
    author={Austin Patel, Shuran Song},
    journal={arXiv preprint arXiv:2407.15002},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.15002},
    primaryClass={cs.RO}
}