Looking at a de-facto reference implementation, it looks like all that needs to be changed is the number of output channels in the last convolution: Pytorch-UNet/unet_model.py at master · milesial/Pytorch-UNet (github.com) 11