If you want to be able to use automatic differentiation, yes. Note that in classical ML, this is also why when your original loss has non-continuous values, you introduce a surrogate loss to be able to optimize something.
Same as 1
If you try to run this code with float tensors that require gradients, you will get the following error: “arguments don’t support automatic differentiation”. So if your code ran fine, it was because the inputs did not require gradients.