Hi,
Pytorch does not use finite difference during the backward pass but automatic differentiation (a chain of analytical functions).
Only the gradcheck tool we use to test our backward pass actually does finite difference. But this is only used for testing purposes.