Unveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph
<|reference_start|>Unveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph: Large Language Models (LLMs) demonstrate an impressive capacity to recall a vast range of factual knowledge. However, understanding their underlying reasoning and internal mechanisms in exploiting this knowledge remains a key research area. This work unveils the factual information an LLM represents internally for sentence-level claim verification. We propose an end-to-end framework to decode factual knowledge embedded in token representations from a vector space to a set of ground predicates, showing its layer-wise evolution using a dynamic knowledge graph. Our framework employs activation patching, a vector-level technique that alters a token representation during inference, to extract encoded knowledge. Accordingly, we neither rely on training nor external models. Using factual and common-sense claims from two claim verification datasets, we showcase interpretability analyses at local and global levels. The local analysis highlights entity centrality in LLM reasoning, from claim-related information and multi-hop reasoning to representation errors causing erroneous evaluation. On the other hand, the global reveals trends in the underlying evolution, such as word-based knowledge evolving into claim-related facts. By interpreting semantics from LLM latent representations and enabling graph-related analyses, this work enhances the understanding of the factual knowledge resolution process.<|reference_end|>
@article{bronzini2024unveiling,
    title={Unveiling LLMs: The Evolution of Latent Representations in a Dynamic
  Knowledge Graph},
    author={Marco Bronzini, Carlo Nicolini, Bruno Lepri, Jacopo Staiano, Andrea
  Passerini},
    journal={arXiv preprint arXiv:2404.03623},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.03623},
    primaryClass={cs.CL cs.AI cs.CY}
}