Found this talk from Alex Graves also to be complementary, he also speaks about transformers more generally with advantages/disadvantages.
The Q/A at the end of these videos I found to be helpful.
Day 6: Lecture - Attention &amp; Memory with Alex Graves