I think layer norm is generally used after nn.Embedding because we do not want to mix one word’s embedding with another word’s embedding while normalizing.
I think you could go with other normalizing technique like batchnorm, if you want to use layernorm after applying conv1d, then you will have to pass size of last dim, that would be

Screenshot%20(198)869×108 6.43 KB