Hi,
I have been wondering how autograd actually works, i.e. how does it compute the gradients? I understand that it does not use any numerical methods (i.e. finite element methods etc).
Are all method’s derivative pre-defined / calculated, i.e. d/dx (x^p) = p * x^(p-1) inside Autograd or is it using some symbolic methods?
Also, do all deep learning framework (i.e. PyTorch, Tensorflow, MxNet etc) use the same method or is there different between them?
Thanks.