Solved by ptrblck in post #2 
              
              
                In your first use case (different number of input channels) you could add a conv layer before the pre-trained model and return 3 out_channels. 
For different input sizes you could have a look at the source code of vgg16. There you could perform some model surgery and add an adaptive pooling layer in…