title: Embodied Agents Meet Personalization: Exploring Memory Utilization for
    Personalized Assistance
summary: Embodied agents empowered by large language models (LLMs) have shown strong
  performance in household object rearrangement tasks. However, these tasks
  primarily focus on single-turn interactions with simplified instructions, which
  do not truly reflect the challenges of providing meaningful assistance to
  users. To provide personalized assistance, embodied agents must understand the
  unique semantics that users assign to the physical world (e.g., favorite cup,
  breakfast routine) by leveraging prior interaction history to interpret
  dynamic, real-world instructions. Yet, the effectiveness of embodied agents in
  utilizing memory for personalized assistance remains largely underexplored. To
  address this gap, we present MEMENTO, a personalized embodied agent evaluation
  framework designed to comprehensively assess memory utilization capabilities to
  provide personalized assistance. Our framework consists of a two-stage memory
  evaluation process design that enables quantifying the impact of memory
  utilization on task performance. This process enables the evaluation of agents'
  understanding of personalized knowledge in object rearrangement tasks by
  focusing on its role in goal interpretation: (1) the ability to identify target
  objects based on personal meaning (object semantics), and (2) the ability to
  infer object-location configurations from consistent user patterns, such as
  routines (user patterns). Our experiments across various LLMs reveal
  significant limitations in memory utilization, with even frontier models like
  GPT-4o experiencing a 30.5% performance drop when required to reference
  multiple memories, particularly in tasks involving user patterns. These
  findings, along with our detailed analyses and case studies, provide valuable
  insights for future research in developing more effective personalized embodied
  agents. Project website: https://connoriginal.github.io/MEMENTO
paper_authors._id: 6835365d2925bc8bb23a57c7
paper_authors.name: Taeyoon Kwon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:50:26.210Z
paper_authors._id: 636b529ef796304dd67d139c
paper_authors.avatarUrl: /avatars/7a64d5095fcb1da558b52ad48177ad76.svg
paper_authors.fullname: Taeyoon Kwon
paper_authors.type: user
paper_authors.user: Connoriginal
paper_authors._id: 6835365d2925bc8bb23a57c8
paper_authors.name: Dongwook Choi
paper_authors._id: 6835365d2925bc8bb23a57c9
paper_authors.name: Sunghwan Kim
paper_authors._id: 6835365d2925bc8bb23a57ca
paper_authors.name: Hyojun Kim
paper_authors._id: 6835365d2925bc8bb23a57cb
paper_authors.name: Seungjun Moon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T08:24:40.306Z
paper_authors._id: 6420f4f55bccaa42484496e5
paper_authors.avatarUrl: /avatars/4996ba26955f8423c946b1ecd3989964.svg
paper_authors.fullname: Seung Jun Moon
paper_authors.type: user
paper_authors.user: Lune-Blue
paper_authors._id: 6835365d2925bc8bb23a57cc
paper_authors.name: Beong-woo Kwak
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T08:24:46.911Z
paper_authors._id: 64b72a408ba7d6c922c73054
paper_authors.avatarUrl: /avatars/6d9797430bc36f05fb950b84aa6a9374.svg
paper_authors.fullname: Beong Woo Kwak
paper_authors.type: user
paper_authors.user: bwookwak
paper_authors._id: 6835365d2925bc8bb23a57cd
paper_authors.name: Kuan-Hao Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T08:24:53.502Z
paper_authors._id: 658a57b4126b8d7eae07b983
paper_authors.avatarUrl: /avatars/8d908cb3da697793564d24206a333782.svg
paper_authors.fullname: Kuan-Hao Huang
paper_authors.type: user
paper_authors.user: ej0cl6
paper_authors._id: 6835365d2925bc8bb23a57ce
paper_authors.name: Jinyoung Yeo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T08:25:01.610Z
paper_authors._id: 682e91865fa8c5df85b3d8e5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/XWTfZoOjCdMnqaFEBBYWe.png
paper_authors.fullname: Jinyoung Yeo
paper_authors.type: user
paper_authors.user: jinyeo
paper_title: Embodied Agents Meet Personalization: Exploring Memory Utilization for
    Personalized Assistance
paper_summary: Embodied agents empowered by large language models (LLMs) have shown strong
  performance in household object rearrangement tasks. However, these tasks
  primarily focus on single-turn interactions with simplified instructions, which
  do not truly reflect the challenges of providing meaningful assistance to
  users. To provide personalized assistance, embodied agents must understand the
  unique semantics that users assign to the physical world (e.g., favorite cup,
  breakfast routine) by leveraging prior interaction history to interpret
  dynamic, real-world instructions. Yet, the effectiveness of embodied agents in
  utilizing memory for personalized assistance remains largely underexplored. To
  address this gap, we present MEMENTO, a personalized embodied agent evaluation
  framework designed to comprehensively assess memory utilization capabilities to
  provide personalized assistance. Our framework consists of a two-stage memory
  evaluation process design that enables quantifying the impact of memory
  utilization on task performance. This process enables the evaluation of agents'
  understanding of personalized knowledge in object rearrangement tasks by
  focusing on its role in goal interpretation: (1) the ability to identify target
  objects based on personal meaning (object semantics), and (2) the ability to
  infer object-location configurations from consistent user patterns, such as
  routines (user patterns). Our experiments across various LLMs reveal
  significant limitations in memory utilization, with even frontier models like
  GPT-4o experiencing a 30.5% performance drop when required to reference
  multiple memories, particularly in tasks involving user patterns. These
  findings, along with our detailed analyses and case studies, provide valuable
  insights for future research in developing more effective personalized embodied
  agents. Project website: https://connoriginal.github.io/MEMENTO
paper_ai_summary: MEMENTO evaluates personalized memory utilization in embodied agents, revealing limitations in understanding user semantics and routines.
paper_ai_keywords: embodied agents large language models (LLMs) object rearrangement tasks user semantics prior interaction history memory utilization personalized assistance goal interpretation object semantics user patterns