I have trained a network with UNet algorithm. The input and output for my network are images with (512 x 512) pixels. I use L1loss for certain epochs and then I change the loss function to customized loss function. Now when I back-propagate with the custom loss, I want to update only certain pixels of the image and keep the other pixels as they are. How do I do it? as I have a convolution layer as my last layer. Incase of linearly connected layers, I can think of manually setting the gradients of few pixels to zero.