Thanks for your helpful reply. I had a look at their abstracts, and do not have a firm opinion on whether attention can fully help us understand what I’m looking for.
Both are good tools for interactive visualization but I want something that provide some quantifiable-ness. For now, I’m using srush way of visualizing attention heatmaps like he did in Annotated Transformer. Since I need to report in the paper, I am looking for static visualizations.
Based on my little interaction with exbert live demo, it can be hard for the reader to distinguish between what both models are looking at (for comparsion purposes).
For my use-case, I want the reader to be able to distinguish what two networks look at and how one is better than other. I hope it makes some sense.