Hi,
You need to use 1 if you need custom gradients for you loss function (if it’s not differentiable or the gradients returned by the autograd are not what you want) or if the autograd backward is too slow because of the size og the forward and you could implement it more efficiently by hand.
If you’re not in these cases, go with 2: it’s simpler and will always return correct gradients !