I will try again.
Let’s assume we describe a net with 2 main blocks:

The main layers.
The final layer which takes output of the previous block and its output size is according to the task.

For instance, for 10 classes classification problem we can do what ever we want in the first block and the second block is a fully connected with output size of 10.
Now imaging we have have Block A1 and block B1.
The net is:
input -> A1 -> B1
Now we have A2 as well and we build a net like:
input -> A1 -> A2 -> B2
But in this case gradients doesn’t affect A1 as much as we want.
So we build:
input -> A1 -> A2 -> B2
             |--> B1

Now we have exit point at B1 and B2.
In training we first do a step of the net input -> A1 -> B1.
Then a step for input -> A1 -> A2 -> B2.
How can we do that in PyTorch?