S is the number of elements; N is the number of batches; E is the number of features (a.k.a. embedding dimension in your case).
If you send input (S, N, 5000) to embedding layer, the output will be in the shape of (S, N, 128). Then, you don’t need to make any changes in order to feed them to the transformer layer. The src_mask is just a square matrix which is used to filter the attention weights.
See example here 165