Thanks for the quick reply.
Running the same code on the CPU leads to the following error
RuntimeError: shape mismatch: value tensor of shape [4, 7] cannot be broadcast to indexing result of shape [3, 7]

Where the “shape[3, 7]” is sometimes “shape [0, 7]” even after making sure that the same seed is used for initialization.
Using a batch size of 4, 7 classes and a sequence length of 3
initial decoder_hidden:
tensor([[[-0.9933,  0.6996,  0.6717,  0.9432],
         [-0.9933,  0.6996,  0.6717,  0.9432],
         [-0.9933,  0.6996,  0.6717,  0.9432],
         [-0.9931,  0.7586, -0.8658,  0.8917]]], grad_fn=<UnsqueezeBackward0>)

initial decoder_input:
tensor([3, 3, 3, 3])

Tensors before applying the loss, shown in the following order:

decoder_output
y
valid

tensor([[-1.7833, -2.8133, -2.5647, -2.7465, -0.7936, -2.4181, -2.4134],
        [-1.7833, -2.8133, -2.5647, -2.7465, -0.7936, -2.4181, -2.4134],
        [-1.7833, -2.8133, -2.5647, -2.7465, -0.7936, -2.4181, -2.4134],
        [-2.0692, -3.0334, -2.1953, -2.6085, -0.7661, -2.5575, -2.3198]],
       grad_fn=<IndexBackward>)
tensor([4, 4, 4, 4])
tensor([True, True, True, True])

tensor([[-1.7946, -2.5081, -2.1118, -2.2311, -1.4011, -1.8884, -2.0692],
        [-1.7946, -2.5081, -2.1118, -2.2311, -1.4011, -1.8884, -2.0692],
        [-1.7946, -2.5081, -2.1118, -2.2311, -1.4011, -1.8884, -2.0692],
        [-2.0122, -2.6849, -1.8347, -2.1308, -1.4038, -2.0658, -1.9153]],
       grad_fn=<IndexBackward>)
tensor([0, 0, 0, 6])
tensor([True, True, True, True])

tensor([[-1.8805, -2.4155, -1.9785, -2.0433, -1.7603, -1.7837, -1.8952],
        [-1.8805, -2.4155, -1.9785, -2.0433, -1.7603, -1.7837, -1.8952],
        [-1.8805, -2.4155, -1.9785, -2.0433, -1.7603, -1.7837, -1.8952],
        [-2.0708, -2.5655, -1.7576, -1.9640, -1.7717, -1.9530, -1.7581]],
       grad_fn=<IndexBackward>)
tensor([2, 2, 2, 2])
tensor([True, True, True, True])