Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representation
<|reference_start|>Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representation: Real2Sim2Real plays a critical role in robotic arm control and reinforcement learning, yet bridging this gap remains a significant challenge due to the complex physical properties of robots and the objects they manipulate. Existing methods lack a comprehensive solution to accurately reconstruct real-world objects with spatial representations and their associated physics attributes.   We propose a Real2Sim pipeline with a hybrid representation model that integrates mesh geometry, 3D Gaussian kernels, and physics attributes to enhance the digital asset representation of robotic arms.   This hybrid representation is implemented through a Gaussian-Mesh-Pixel binding technique, which establishes an isomorphic mapping between mesh vertices and Gaussian models. This enables a fully differentiable rendering pipeline that can be optimized through numerical solvers, achieves high-fidelity rendering via Gaussian Splatting, and facilitates physically plausible simulation of the robotic arm's interaction with its environment using mesh-based methods.   The code,full presentation and datasets will be made publicly available at our website https://robostudioapp.com<|reference_end|>
@article{lou2024robo-gs:,
    title={Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm
  with Hybrid Representation},
    author={Haozhe Lou, Yurong Liu, Yike Pan, Yiran Geng, Jianteng Chen, Wenlong
  Ma, Chenglong Li, Lin Wang, Hengzhen Feng, Lu Shi, Liyi Luo, Yongliang Shi},
    journal={arXiv preprint arXiv:2408.14873},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.14873},
    primaryClass={cs.RO cs.NA math.NA math.OC}
}