Hello all, I have a network that contains a base and a refiner component. The base generates coarse image output and the refiner takes the coarse output and improves it. I want to train both modules end-to-end. I wonder how I can do loss calculation in this case?
Is it ok if I just do:
loss = loss_func(coarse, ground_truth) + loss_func(refined, ground_truth) 
loss.backward()
optimizer.step()

If not what will be an efficient alternative? Thank you!