LOP-Field: Brain-inspired Layout-Object-Position Fields for Robotic Scene Understanding
<|reference_start|>LOP-Field: Brain-inspired Layout-Object-Position Fields for Robotic Scene Understanding: Spatial cognition empowers animals with remarkably efficient navigation abilities, largely depending on the scene-level understanding of spatial environments. Recently, it has been found that a neural population in the postrhinal cortex of rat brains is more strongly tuned to the spatial layout rather than objects in a scene. Inspired by the representations of spatial layout in local scenes to encode different regions separately, we proposed LOP-Field that realizes the Layout-Object-Position(LOP) association to model the hierarchical representations for robotic scene understanding. Powered by foundation models and implicit scene representation, a neural field is implemented as a scene memory for robots, storing a queryable representation of scenes with position-wise, object-wise, and layout-wise information. To validate the built LOP association, the model is tested to infer region information from 3D positions with quantitative metrics, achieving an average accuracy of more than 88\%. It is also shown that the proposed method using region information can achieve improved object and view localization results with text and RGB input compared to state-of-the-art localization methods.<|reference_end|>
@article{hou2024lop-field:,
    title={LOP-Field: Brain-inspired Layout-Object-Position Fields for Robotic
  Scene Understanding},
    author={Jiawei Hou, Wenhao Guan, Xiangyang Xue, Taiping Zeng},
    journal={arXiv preprint arXiv:2406.05985},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.05985},
    primaryClass={cs.RO}
}