In the first use case the middle_linear layer will be used a few times with the same weights, since its output is fed into the layer again.
In the second example, the extra_layer will be registered as a parameter in the model but won’t be updated, since the forward method doesn’t use it. You can have a look at its gradients after a backward pass and you’ll see it’s None.
The forward method defines the computation graph every time the model is called. This graph is then used in the backward pass. Since it’ll be created in each forward pass, you can define your computation e.g. using conditions etc. That’s why your model architecture might even change based on some conditions.
Usually you register all parameters in the __init__ method, which are used in the forward method.