thanks, i am having some issues with mult-gpu training. i have multiple encoders, and the data parallel module is splitting along different dimensions for different encoders, so i get error in decoder.
does data parallel module assume the first dimension is the batch? LSTM module assumes Batch is the middle dimension.