Questions for you:

Why couldn’t you just load target labels as indices? This will be more memory efficient and arguably run faster in calculating cross entropy loss.






 samlanka:

I then had to use .squeeze() because .view() was not sufficient and I also had to typecast from FloatTensor to LongTensor.


Isn’t it nature that you would get LongTensor when converting one-hot to label indices? Every straightforward way to convert in PyTorch I can think of gives this property, e.g. torch.max.
3)



 samlanka:

I then had to use .squeeze() because .view() was not sufficient


.view can always do what .squeeze do.
4)



 samlanka:

I had to first flatten my input to a 2D tensor of [seq len*batchsize x one hot dim] and had to convert my target from one-hot to a suitable 1D tensor.


You don’t need to “flatten” your input. No where in the doc says that http://pytorch.org/docs/master/nn.html#recurrent-layers 5, and RNN modules always only accepts 3D input I believe. Simply set batch_first=True should be fine.
Moreover, since data dim (i.e. one hot dim) is last dimension, shouldn’t it actually be very simple and straightforward? Something along the lines of
loss = criterion(out.view(-1, data_dim), labels.max(-1)[1].view(-1))

should just work.