Larimar: Large Language Models with Episodic Memory Control
Efficient and accurate updating of knowledge stored in Large Language Models
(LLMs) is one of the most pressing research challenges today. This paper
presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with
a distributed episodic memory. Larimar's memory allows for dynamic, one-shot
updates of knowledge without the need for computationally expensive re-training
or fine-tuning. Experimental results on multiple fact editing benchmarks
demonstrate that Larimar attains accuracy comparable to most competitive
baselines, even in the challenging sequential editing setup, but also excels in
speed - yielding speed-ups of 4-10x depending on the base LLM - as well as
flexibility due to the proposed architecture being simple, LLM-agnostic, and
hence general. We further provide mechanisms for selective fact forgetting and
input context length generalization with Larimar and show their effectiveness.
65f9194a9622bdda62c68f76
Payel Das
admin_assigned
2024-03-19T10:08:44.291Z
62d8314f3cd901df9443bceb
/avatars/3556ea20a032651db820190aa6508217.svg
Payel Das
user
pdas79
65f9194a9622bdda62c68f77
Subhajit Chaudhury
admin_assigned
2024-03-19T10:09:10.308Z
63536559a023833e59da3458
/avatars/bcf1c4193f75bb464998a02403f1eb91.svg
Subhajit Chaudhury
user
cosmik1412
65f9194a9622bdda62c68f78
Elliot Nelson
admin_assigned
2024-03-19T10:09:18.067Z
635848c1856b319a29b9a9cc
/avatars/a797c1c644cdc819d9bfe411b4c0d6f5.svg
Elliot Nelson
user
enelson
65f9194a9622bdda62c68f79
Igor Melnyk
65f9194a9622bdda62c68f7a
Sarath Swaminathan
admin_assigned
2024-03-19T10:09:49.509Z
63521353beef5fb9913b69c5
/avatars/349c59f22f43e2b1aa4205a34ab2f824.svg
Sarath Swaminathan
user
sarathkrishna
65f9194a9622bdda62c68f7b
Sihui Dai
65f9194a9622bdda62c68f7c
Aurélie Lozano
65f9194a9622bdda62c68f7d
Georgios Kollias
65f9194a9622bdda62c68f7e
Vijil Chenthamarakshan
admin_assigned
2024-03-19T10:10:15.161Z
6054af4634c3e14ef093c0a8
/avatars/954780f7bbcf1a4aa30fb31c2442c2c8.svg
Vijil Chenthamarakshan
user
thunderingtyphoons
65f9194a9622bdda62c68f7f
Jiří
65f9194a9622bdda62c68f80
Navrátil
65f9194a9622bdda62c68f81
Soham Dan
admin_assigned
2024-03-19T10:10:24.819Z
65f61702133b39d44f3160f6
/avatars/320185b4c2ea51818c946fb746b948df.svg
Soham Dan
user
soham06
65f9194a9622bdda62c68f82
Pin-Yu Chen
admin_assigned
2024-03-19T10:10:35.662Z
6495dd0b71f6708e0f990032
https://cdn-avatars.huggingface.co/v1/production/uploads/6495dd0b71f6708e0f990032/PBIjdKNnpkxvR_3djCGVm.png
Pin-Yu Chen
user
pinyuchen
Larimar: Large Language Models with Episodic Memory Control
Efficient and accurate updating of knowledge stored in Large Language Models
(LLMs) is one of the most pressing research challenges today. This paper
presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with
a distributed episodic memory. Larimar's memory allows for dynamic, one-shot
updates of knowledge without the need for computationally expensive re-training
or fine-tuning. Experimental results on multiple fact editing benchmarks
demonstrate that Larimar attains accuracy comparable to most competitive
baselines, even in the challenging sequential editing setup, but also excels in
speed - yielding speed-ups of 4-10x depending on the base LLM - as well as
flexibility due to the proposed architecture being simple, LLM-agnostic, and
hence general. We further provide mechanisms for selective fact forgetting and
input context length generalization with Larimar and show their effectiveness.
Larimar, a brain-inspired architecture with distributed episodic memory, enhances LLMs for efficient, fast, and accurate knowledge updates, fact editing, and context length generalization.
Larimar brain-inspired architecture distributed episodic memory knowledge updates one-shot updates fact editing selective fact forgetting input context length generalization LLM-agnostic sequential editing speed-ups