Figure 10: Attention visualization for a dialogue with and without GAtt. We considered the maximum
activations across the network and we bin neighboring tokens together.
To illustrate how GAtt helped reshape attention during fine-tuning, we display the maximum attention
activations of the model in Figure 10. The left-hand side of each figure corresponds to the system message
(“Act as Oscar Wilde”). We can see that the GAtt-equipped model (right) maintains large attention activations
with respect to the system message for a larger portion of the dialogue, as compared to the model without
GAtt (left).
Despite its utility, the current implementation of GAtt is vanilla, and more development and iteration on
this technique could likely further benefit the model. For instance, we could teach the model to change the
system message during the conversation by integrating such data during fine-tuning.
Llama2