The original UNet paper 3 uses a similar strategy, i.e. a sliding window approach where only the center is predicted and mirrored pixel values on the border, if I’m not mistaken.
Did you see any improvement if you predicted the center of the sliding window compared to the complete patch?