I am implementing a encoder-decoder transformer for language translation and using the TensorFlow tutorial as a guide: Transformer model for language understanding  |  Text  |  TensorFlow 3
I am confused at a specific point, notably where the MultiHead Class is created. There seems to be a discrepancy between the schematic figure: On the figure the q,k,v are first split and then run through the Dense layer. However, in the code itself, it seems they are first run through the dense layer, and only afterwards split up.
This is a significant difference, right?
Thanks