Even more puzzling is when I check if the two tensors are equal, i.e.,
a = encoder_outputs[ei]+encoder_output[0][0]
b = encoder_output[0][0]
torch.equal(a,b)
As expected, they are equal for any input  so, I am really baffled now