  The development of embodied agents that can communicate with humans in
natural language has gained increasing interest over the last years, as it
facilitates the diffusion of robotic platforms in human-populated environments.
As a step towards this objective, in this work, we tackle a setting for visual
navigation in which an autonomous agent needs to explore and map an unseen
indoor environment while portraying interesting scenes with natural language
descriptions. To this end, we propose and evaluate an approach that combines
recent advances in visual robotic exploration and image captioning on images
generated through agent-environment interaction. Our approach can generate
smart scene descriptions that maximize semantic knowledge of the environment
and avoid repetitions. Further, such descriptions offer user-understandable
insights into the robot's representation of the environment by highlighting the
prominent objects and the correlation between them as encountered during the
exploration. To quantitatively assess the performance of the proposed approach,
we also devise a specific score that takes into account both exploration and
description skills. The experiments carried out on both photorealistic
simulated environments and real-world ones demonstrate that our approach can
effectively describe the robot's point of view during exploration, improving
the human-friendly interpretability of its observations.

Robotics;Computer Vision and Pattern Recognition