You can load the pretrained model and then take apart the network and add your classifiers and have these outputs returned in your forward pass.  You can take apart the network by iterating through named_children or children.