Question
I would like to implement a network architecture similar to Inception shown below. They are similar in the sense that there are multiple half-way branches (the part within in blue block).

image.png828×301 61 KB

In order to implement this architecture, I could write those HalfWayBranch within MainBranch. However, I would like different modules to be least entangled together.
Then I have
class MainBranch(nn.Module):
# ...

class HalfWayBranch(nn.Module):
# ...

Now I have two questions

How do I write my training loop?
Could the loss be backproped correctly?