I see. So just to be clear then: Does this mean that the very first attention layer of the transformer needs an input of dimension d_embed, and the second attention layer expects an input of dimensions d_inner?
Edit: re-reading the original “attention is all you need” paper, I realize that there are actually two feed-forward layers, and that d_inner is the dimension of the first one. The second one bring the input to the correct dimension of d_embed again.