Hi ptrblck,
Thanks for the explicit solution. After I tried that it also seemed not working. I am trying to put nn.DataParallel on the whole SinusoidalPositionalEmbedding object and see what will happen.