Patchscope: A Unifying Framework for Inspecting Hidden Representations
  of Language Models
Inspecting the information encoded in hidden representations of large
language models (LLMs) can explain models' behavior and verify their alignment
with human values. Given the capabilities of LLMs in generating
human-understandable text, we propose leveraging the model itself to explain
its internal representations in natural language. We introduce a framework
called Patchscopes and show how it can be used to answer a wide range of
research questions about an LLM's computation. We show that prior
interpretability methods based on projecting representations into the
vocabulary space and intervening on the LLM computation, can be viewed as
special instances of this framework. Moreover, several of their shortcomings
such as failure in inspecting early layers or lack of expressivity can be
mitigated by a Patchscope. Beyond unifying prior inspection techniques,
Patchscopes also opens up new possibilities such as using a more capable model
to explain the representations of a smaller model, and unlocks new applications
such as self-correction in multi-hop reasoning.
65a0a8ae33d5d9ca30dbb594
Asma Ghandeharioun
claimed_verified
2024-01-12T08:06:29.260Z
6501fca46aac47d19ab1b17a
/avatars/1685b6b1f2e9329d92b53e1070d8c7c4.svg
AG
user
asmadotgh
65a0a8ae33d5d9ca30dbb595
Avi Caciularu
admin_assigned
2024-01-12T09:38:58.246Z
6033d0681f993496bc14d9eb
/avatars/1c0dfc1fe7b62a78b76ec4f9e6b40b24.svg
Avi Caciularu
user
codevan
65a0a8ae33d5d9ca30dbb596
Adam Pearce
admin_assigned
2024-01-12T09:40:17.136Z
6266c1f5ca8ef0cabfc20e9f
/avatars/78310b60a2cd461a4b980a339a677c7c.svg
Adam Pearce
user
1wheel
65a0a8ae33d5d9ca30dbb597
Lucas Dixon
admin_assigned
2024-01-12T09:40:23.035Z
64c25a9c2b1ffc2eecbce31d
https://cdn-avatars.huggingface.co/v1/production/uploads/64c25a9c2b1ffc2eecbce31d/uTp3yOPdmCSAmhzWwKUjK.png
Lucas Dixon
user
iislucas
65a0a8ae33d5d9ca30dbb598
Mor Geva
claimed_verified
2024-01-12T08:06:25.609Z
610b729f9da682cd54ad9adf
https://cdn-avatars.huggingface.co/v1/production/uploads/1628140189042-noauth.jpeg
Mor Geva
user
mega
Patchscope: A Unifying Framework for Inspecting Hidden Representations
  of Language Models
Inspecting the information encoded in hidden representations of large
language models (LLMs) can explain models' behavior and verify their alignment
with human values. Given the capabilities of LLMs in generating
human-understandable text, we propose leveraging the model itself to explain
its internal representations in natural language. We introduce a framework
called Patchscopes and show how it can be used to answer a wide range of
research questions about an LLM's computation. We show that prior
interpretability methods based on projecting representations into the
vocabulary space and intervening on the LLM computation, can be viewed as
special instances of this framework. Moreover, several of their shortcomings
such as failure in inspecting early layers or lack of expressivity can be
mitigated by a Patchscope. Beyond unifying prior inspection techniques,
Patchscopes also opens up new possibilities such as using a more capable model
to explain the representations of a smaller model, and unlocks new applications
such as self-correction in multi-hop reasoning.
Patchscopes is a framework that uses LLMs to explain their internal representations, unifying and improving upon prior interpretability methods and enabling new applications like self-correction.
large language models hidden representations model behavior Patchscopes interpretability methods vocabulary space early layers expressivity self-correction multi-hop reasoning