BramVanroy:

The blue cells are not individual encoders either, each row of blue circles is a single encoder layer


This was the answer I was looking for, many thanks.
Initially, I was thinking in this way, however, in the BERT paper there is a comparison between it and the old state of the art models, in particular with GPT (which I have not studied).

Looking at the image I imagined that each blue circle was effectively a separated Transformer.
Then, following your logic, in the case of GPT,  in the first multihead attention layer only the first token is given in input, in the second layer, the first two and so on.
Is that correct?