Thanks for your reply. By changing CrossEntropyLoss to MSELoss the problem was solved. For your interest, I can explain more details that I didn’t specify clearly in the original post. Essentially, the project tries to map some words that a person speaks to a video of the person moving his lips. So the audio features are MFCC of the audio signal, while the video features are parameters that measure how the lip on each frame differs from the original image(a closed mouth) so I can create different images of the lip to make a video. I converted the target to torch.long because the CrossEntropyLoss requires the argument to be LongTensor otherwise there is an error. By changing the criterion to MSELoss I don’t need to worry about it. Really appreciate your answer!
Just as a follow-up, since I am using MSE loss to measure the accuracy, I can no longer calculate the accuracy by dividing total samples by correct predictions. Is there a reasonable metric/function to measure the performance of the model when the loss function is MSE Loss?