Padding tensors with zeros will affect the loss calculation and you would have to think about the actual loss calculation and what it should represent.
Since the model output and targets have different shapes you won’t be able to directly calculate the loss with e.g. MSELoss so could you explain what the shapes in both tensors represent?