Hi all,
I have been reading up on differentiation techniques in DL frameworks. I came across the terms symbol-to-number differentiation which is used by Caffe and symbol-to-symbol differentiation. The former describes the approach of setting actual numeric values to the computational graph and computing the derivative in terms of those numeric values. The later describes an approach of adding additional nodes to the computational graph (providing a description of the derivates) which are used to compute the gradients. This later approach is used by TensorFlow. I was wondering which approach PyTorch here takes.
Best and thank you.