wwaayyaaww:

What would you suggest if I am to switch between the two linear layers?


One approach would be to pass a flag to the forward method and use it as a condition to switch between the linear heads in your model.