just create new layers on the fly behind python conditionals etc.
pytorch’s models are python code, so it’s upto you how you want to do these things. your forward function gives you full freedom to do arbitrary things.