You can also implement models with attention, including the Transformer-based models. Here are some examples:

Neural machine translation with attention  |  Text  |  TensorFlow 2
Transformer model for language understanding  |  Text  |  TensorFlow 3
Text classification with Transformer
Image classification with Vision Transformer 2

API docs, including:

tf.keras.layers.Attention  |  TensorFlow Core v2.5.0
tf.keras.layers.MultiHeadAttention  |  TensorFlow Core v2.5.0 2