Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
function to access the weights of the attention heads : fig = plt.figure(figsize=(10., 10.)) images = model.decoder.decoder_blocks[0].cross_attention.attention_weigths[0,...].detach().numpy() grid = ImageGrid(fig, 111, # similar to subplot(111) nrows_ncols=(2, 2), # creates 2x2 grid of axes