Hi,
I am going through the advanced pytorch tutorial on image-captioning.

  
      
      GitHub
  
  
    

yunjey/pytorch-tutorial 60

pytorch-tutorial - PyTorch Tutorial for Deep Learning Researchers


  
  
    
    
  
  


The training module combines the parameters of CNN and RNNs as follow and give them to optimizer.
criterion = nn.CrossEntropyLoss()
params = list(decoder.parameters()) + list(encoder.linear.parameters()) + list(encoder.bn.parameters())
optimizer = torch.optim.Adam(params, lr=args.learning_rate)
# Train the Models
total_step = len(data_loader)
for epoch in range(args.num_epochs):
    for i, (images, captions, lengths) in enumerate(data_loader):
        
        # Set mini-batch dataset
        images = to_var(images, volatile=True)
        captions = to_var(captions)
        targets = pack_padded_sequence(captions, lengths, batch_first=True)[0]
        
        # Forward, Backward and Optimize
        decoder.zero_grad()
        encoder.zero_grad()
        features = encoder(images)
        outputs = decoder(features, captions, lengths)
        loss = criterion(outputs, targets)
        loss.backward()
        optimizer.step()

Is it considered End-To-End training? Apparently it seems that the CNN and RNNs are being trained separatly to minimize the loss function. I am not able to see how the gradients from RNNs flow back to CNN. Does the pytorch takes care of the flow of gradient from RNN to CNN or it just uses the gradient of the loss and update all the parameters at once?
Also is there any example that implements the backward pass in pytorch for encoder-decoder architectures?