Solved by albanD in post #2 
              
              
                Hi, 
That will work fine. You can directly do: 
common_layer = f(input)

branch1 = f(common_layer)

branch2 = f(common_layer)

output = a*branch1 + (1-a)*banch2

As long as f is an nn.Module with no funky hooks. Or a simple python function.