Hi @tom
Thank you for the insights – to your first point, I did incorrectly use the $i$ index twice, my mistake
I think a permutation invariant function for aggregating attentions as suggested in your first point would likely be the easiest approach.
I’m not sure I understand the “external attention” approach, could you elaborate a bit here?
k