Solved by ptrblck in post #2 
              
              
                Since the last convolution layer has only a single channel, you won’t be able to use it for 3 classes. 
Instead you would have to assign a new nn.Conv2d layer with out_channels=3 to model.final and retrain this layer for your new use case.