Solved by ptrblck in post #4 
              
              
                You are recreating the tensor here without any gradient history: 
currCriticVals = torch.tensor(currCriticVals[0:advantages.shape[0]], dtype=torch.float, requires_grad=True, device=device)

currCriticVals is a new tensor, which requires gradients without any history and will thus detach the computat…