The main advantage of pytorch is actually that your forward function can be any python function that works with pytorch’s Variables. So you can create a layer that does anything you want, including using other nn.Modules. Basically as long as you don’t get an error that tells you that you’re doing something forbidden, then it works.