  This study introduces a novel approach for analyzing and modifying entity
relationships in GPT models, diverging from ROME's entity-focused methods. We
develop a relation tracing technique to understand the influence of language
model computations on relationship judgments. Using the FewRel dataset, we
identify key roles of MLP modules and attention mechanisms in processing
relationship information. Our method, tested against ROME on a new dataset,
shows improved balance in specificity and generalization, underscoring the
potential of manipulating early-layer modules for enhanced model understanding
and accuracy.

Computation and Language;Artificial Intelligence