Hi,
I want to clarify that my implementation is correct, I have not used attention yet so I unroll decoder in one call:
everywhere I use batch_first=True
consider simple case: batch_size=2, hidden_size=4, len(vocab) = 10
I pad every sequence(sentence) to max_ length so in our simple case decoder input:
1    6    9    9    4
1    9    9    9    0
[torch.LongTensor of size 2x5]

Outputs from nn.LSTM in the decoder have such format:
Variable containing:
(0 ,.,.) = 
  0.1351 -0.0738 -0.3071  0.1253
  0.2045  0.0473 -0.4745  0.0952
  0.1976  0.1333 -0.1086  0.0051
  0.1840  0.1820 -0.1250  0.0794
  0.2153  0.1870 -0.0804  0.0017

(1 ,.,.) = 
  0.1388 -0.0739 -0.3141  0.4524
  0.2480 -0.0281 -0.3296  0.3183
  0.2284  0.0410 -0.1947  0.1689
  0.2259  0.0712 -0.1931  0.1656
  0.2268  0.0772 -0.0793  0.3217
[torch.cuda.FloatTensor of size 2x5x4 (GPU 0)]

Then I create output_mask because I don’t want to compute pad elements:
Variable containing:
(0 ,.,.) = 
  1  1  1  1
  1  1  1  1
  1  1  1  1
  1  1  1  1
  1  1  1  1

(1 ,.,.) = 
  1  1  1  1
  1  1  1  1
  1  1  1  1
  1  1  1  1
  0  0  0  0
[torch.cuda.ByteTensor of size 2x5x4 (GPU 0)]

After it I use torch.masked_select and compute masked_outputs and get:
Variable containing:
 0.1351
-0.0738
-0.3071
 0.1253
 0.2045
 0.0473
-0.4745
 0.0952
 0.1976
 0.1333
-0.1086
 0.0051
 0.1840
 0.1820
-0.1250
 0.0794
 0.2153
 0.1870
-0.0804
 0.0017
 0.1388
-0.0739
-0.3141
 0.4524
 0.2480
-0.0281
-0.3296
 0.3183
 0.2284
 0.0410
-0.1947
 0.1689
 0.2259
 0.0712
-0.1931
 0.1656
[torch.cuda.FloatTensor of size 36 (GPU 0)]

Then I use masked_outputs.view((-1, hidden_size)) and get:
Variable containing:
 0.1351 -0.0738 -0.3071  0.1253
 0.2045  0.0473 -0.4745  0.0952
 0.1976  0.1333 -0.1086  0.0051
 0.1840  0.1820 -0.1250  0.0794
 0.2153  0.1870 -0.0804  0.0017
 0.1388 -0.0739 -0.3141  0.4524
 0.2480 -0.0281 -0.3296  0.3183
 0.2284  0.0410 -0.1947  0.1689
 0.2259  0.0712 -0.1931  0.1656
[torch.cuda.FloatTensor of size 9x4 (GPU 0)]

After all I use nn.Linear(hidden_size, len(vocab)) and get outputs from the decoder:
Variable containing:
 0.4705 -0.0552  0.4348 -0.0798  0.0775  0.3475  0.2021  0.6573 -0.0601  0.2252
 0.4778  0.0825  0.5056 -0.0496  0.1685  0.3090  0.2437  0.7343 -0.0047  0.1911
 0.3115 -0.0023  0.3454 -0.2025  0.1078  0.3741  0.2473  0.5391  0.0743  0.2276
 0.2770 -0.0017  0.3527 -0.2080  0.1067  0.3662  0.2493  0.5297  0.0664  0.2271
 0.2695  0.0146  0.3323 -0.2272  0.1143  0.3801  0.2603  0.5193  0.1007  0.2303
 0.3459 -0.1172  0.4360 -0.0959 -0.0210  0.4020  0.2096  0.6514 -0.1421  0.2980
 0.3347 -0.0184  0.4390 -0.1099  0.0262  0.4214  0.2549  0.6870 -0.0614  0.3002
 0.3110 -0.0216  0.3812 -0.1647  0.0529  0.4106  0.2519  0.6053  0.0044  0.2746
 0.3005 -0.0126  0.3806 -0.1708  0.0629  0.4036  0.2545  0.5973  0.0147  0.2681
[torch.cuda.FloatTensor of size 9x10 (GPU 0)]

I need decoder targets to compute loss so decoder_targets:
6  9  9  4  2
9  9  9  2  0
[torch.LongTensor of size 2x5]

Then I also use torch.masked_select for decoder_targets and get masked_targets:
6
9
9
4
2
9
9
9
2
[torch.cuda.LongTensor of size 9 (GPU 0)]

In the end I compute loss:
loss = criterion(decoder_outputs, masked_targets)
Is this correct implementation for the decoder? Can I use such approach?
Thanks!