Hey ! Thanks a lot for your answer.
I’m fine with backprop. Mostly my problem was that I just wasn’t sure whether the computation graph was sufficiently explicit for PyTorch to manage attributing the gradients given the loss was mixed.
Then, I guess that in REINFORCE with baseline the model should also share some weights, shouldn’t it ?
Thanks again !