Based on your description, the error is raised since nn.SmoothL1Loss expects the output and target to have the same shape.
If you want to use a batch size of 1 and use 3 different images with 17 points each, you could make sure the output and target have the shape [batch_size, 3, 17] or you could also flatten the points to [batch_size, 51]. The calculated loss would be the same.



 ahmed:

Is there a way that I can combine/reshape cat_kp so that it is torch.Size([51, 1]) instead of torch.Size([51, 3]) without losing too much feature quality?


I don’t know, as I’m not familiar with your use case.
However, I would recommend to fix the shape issues and keep the model “idea” unchanged for now.