While implementing neural models by myself without aid of pytorch module, I become curious about how backpropagation is implemented in pytorch.
In pytorch, explicit implementation of backward function is not needed unlike forward function.
However, I can’t understand how pytorch automatically understands the structure of a net so that backpropagation can be carried out.
For example, let’s see the picture below.

It can be implemented like this.
maxpool1 = nn.MaxPool2d((7, 1), stride=(1, 1))
maxpool2 = nn.MaxPool2d((7, 1), stride=(1, 1))
m1 = maxpool1.forward(x1)
m2 = maxpool2.forward(x2)
c = m1 + m2

pytorch must know that there is an add layer to convey dL to the maxpool Layer, but c is calculated by just adding two, without a certain layer class.
(Like this)
add = nn.some_add_layer(2) # 2 inputs
c = add.forward(m1, m2)

Then how does pytorch know that there is an add layer?