If I understand your use case correctly, you would like to add these auxiliary modules into your model.
I would recommend to write a main module and initialize the base model as well as these aux modules inside the __init__.
Then in the forward you could simply use an intermediate output, feed it to the next module in your base model as well as one of the aux modules, and return all necessary outputs at the end. This would be similar to the Inception implementation 12.