Hi Lewis,
your solution works great and its really impressiv how easy it is to generate a good prediction accuracy 
I found an extremly good intro to transformers 44 and thinks are adding up for me now - so, I can create a TF version based on your code.
So, thanx a bunch for getting me started     
Dirk