If the target image is scaled by an integer factor (in your case 2x), you could increase the kernel size and stride by the same factor in the unfold method, which should create the same number of patches for the input and target (of course with a different size).