Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
should be able to see the following : And if we print the attention head we will observe the following : fig = plt.figure() images = model.decoder.decoder_blocks[0].cross_attention.attention_weigths[0,...].detach().numpy().mean(axis=0) fig, ax = plt.subplots(1,1, figsize=(10., 10.)) # Iterating