Yes, you’ve to reshape patches to regain spatial dimensions,
patches.reshape(3, p_size, p_size, -1).permute(3, 0, 1, 2)

the output shape will be the number of patches x 3 x tile_size x tile_size. I used this code for segmentation because I had also the GPU constraint so I applied my model patchwise.
The code you posted worked for me after splitting and rebuilding a test image, so my guess would be either (i) you have to renormalize each tile/image (you could test that by saving the patches before rebuilding them with torch.utils.save_image(…, normalize=True)) or your models causes the problem, maybe you should try to have some overlap/stride to get ride of the boundaries.