Hi,
PyTorch uses automatic differentiation to compute all the gradients. See here 30 for more info about AD.

Also, does it calculate the derivative of non-differentiable functions?

For functions that are non-differentiable only at a single point, we extend the gradient with either a subgradient or by continuity.
For fonctions that are more generally non-differentiable, it will raise an error if you try to give a Tensor that requires gradients to them.