Right now, the optimizer expects a LongTensor with a target

You are confusing loss function for optimizer. If you want, change the loss function (what you say about LongTensor is specific to NLLLoss or CrossEntropyLoss)