Hello,
I am really sorry for this question which is probably trivial to answer but I am struggling to find an answer and I am a beginner.
I am trying to implement a simple autoencoder following the RedNet architecture designed for denoising (I know there exists many implementations but it is just a way for me to understand autoencoder). The encoder and decoder are symetric (stride: 2 for the encoder / upsample X2 for  the decoder) and this architecture uses skip layers: the output of some layers of the encoder are directly summed with the corresponding output of the decoder (i.e the output of the first deconvolution will be summed with the output of the second last deconvolution layer of the decoder).
Here is my code (for the encoder decoder):
def encoder(self, features):

    x = self.enc_conv_1(features)
    fMap1 = F.leaky_relu(x) 
   
    x = self.enc_conv_2(fMap1)
    fMap2 = F.leaky_relu(x)
    
    x = self.enc_conv_3(fMap2)
    fMap3 = F.leaky_relu(x)

    encoded = self.enc_linear(fMap3)

    return encoded, fMap1, fMap2

def decoder(self, encoded, fMap1, fMap2):
    x = self.dec_linear_1(encoded)
    x = x.view(-1, 64, 64, 64)

    x = self.dec_deconv_1(x)
    x = x + fMap2
    x = F.leaky_relu(x)

    x = self.dec_deconv_2(x)
    x = x + fMap1
    x = F.leaky_relu(fx)

    x = self.dec_deconv_3(x)
    x = F.leaky_relu(x)

    decoded = torch.sigmoid(x)
    return decoded

def forward(self, features):
    
    encoded, fMap1, fMap2 = self.encoder(features)
    decoded = self.decoder(encoded,fMap1, fMap2)
     
    return decoded

My question is, when gradients are backpropagated will the autograd mechanism also backpropate gradients directly through the “skip path”, for instance, will the gradient backpropagate through all the network but also diretly from the second last layer of the decoder back to the first layer of the encoder?
If not, how can this be done, do I need to merge the encoder and decoder function within the same function?
I hope my question is clear. I thank you for your help.
Kind regards
Francois