Arham_Khan:

output = net(torch.reshape(torch.LongTensor(curr_seq).to(device), (1,1,seq_len)))


You are feeding the same input curr_seq  in each step.
The Tanh() function I added is a design choice adopted from Deep Voice 3.