SofiaCP:

Either way, would something like this work during backprop? The idea is that during backprop the common trunk is updated by all losses but the subbranches are only updated by the losses of their outputs.


Yes this would work. For example, the following work seems to follow this paradigm.
ensemblenet: end-to-end optimization of multi-headed models 3



 SofiaCP:

I’m not sure how to do that


This might help.
  

      gist.github.com
  

  
    https://gist.github.com/InnovArul/871cf866b737b306eb073f3c37a6a8fd 2

  reset_params.py
  import torchvision, copy
import torch, torch.nn as nn

def reset_all_weights(model: nn.Module) -> None:
    """
    refs:
        - https://discuss.pytorch.org/t/how-to-re-set-alll-parameters-in-a-network/20819/6
        - https://stackoverflow.com/questions/63627997/reset-parameters-of-a-neural-network-in-pytorch
        - https://pytorch.org/docs/stable/generated/torch.nn.Module.html
    """
    This file has been truncated. show original