torch.cat expects all dimension to have the same shape besides the one which is used to concatenate the tensors. Based on the error message you are hitting a shape mismatch in data.pos and pixel_values. Based on the code I guess the mismatch is caused in dim0 and you are trying to concatenate the tensors in dim1. This line of code looks related: pixel_values = data.pixel_values.view(-1, 3) as you are reshaping the pixel_values tensor right before this op so check what shape is expected.