If I understood well, I think that you can do that by simply write something like this in the forward function
def forward(self, x):
        previous = x
        x = self.layer_or_model(x)
        x += previous
        return x

and use it as a building block. I believe this kind of solution works because I used it before but I am not sure if this is what you asked for.