OK. I fixed the error. I had to unsqueeze the tensor eventually but that was not the only problem. I actually tried the unsqueeze(input, 0) solution before but it still did not work because one of the thing in the train part was set it wrong which was this:

    real_cpu = data[0].to(device=device, dtype=torch.float)


This is set to [0]. After I removed the [0] and unsqueeze the batch, the model worked!