EmBARDiment: an Embodied AI Agent for Productivity in XR
<|reference_start|>EmBARDiment: an Embodied AI Agent for Productivity in XR: XR devices running chat-bots powered by Large Language Models (LLMs) have tremendous potential as always-on agents that can enable much better productivity scenarios. However, screen based chat-bots do not take advantage of the the full-suite of natural inputs available in XR, including inward facing sensor data, instead they over-rely on explicit voice or text prompts, sometimes paired with multi-modal data dropped as part of the query. We propose a solution that leverages an attention framework that derives context implicitly from user actions, eye-gaze, and contextual memory within the XR environment. This minimizes the need for engineered explicit prompts, fostering grounded and intuitive interactions that glean user insights for the chat-bot. Our user studies demonstrate the imminent feasibility and transformative potential of our approach to streamline user interaction in XR with chat-bots, while offering insights for the design of future XR-embodied LLM agents.<|reference_end|>
@article{bovo2024embardiment:,
    title={EmBARDiment: an Embodied AI Agent for Productivity in XR},
    author={Riccardo Bovo, Steven Abreu, Karan Ahuja, Eric J Gonzalez, Li-Te
  Cheng, Mar Gonzalez-Franco},
    journal={arXiv preprint arXiv:2408.08158},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.08158},
    primaryClass={cs.HC cs.MA}
}