Yes, this should be the case, since the optimizers should check if the gradients are even set (e.g. here 3 for Adam). In any case, I would always double check the behavior especially if you are using custom optimizers, as they might use another workflow.