Thanks, 


I am sending max_len x batch size x embedding_size input to transformer layer. This is the same api as in LSTM (without batch_first=True). Since, data parallel module expects the module’s input to have batch in first dimension, I am passing batch as first dimension and then reshaping it before passing it to transformer.


Same for images, batch_size x embedding_size input is passed to the module.


Do you see anything wrong here?