From the doc about Function 's forward() , it says “Keep in mind that only Variable s will be passed in here” (http://pytorch.org/docs/0.1.12/notes/extending.html 7), and after reading many posts about create custom function/loss, I have the conclusion that operations on Variables will be able to get gradients through autograd, then I am wondering why it still requires backward() implementation for Functions if inputs to forward() are Variables rather than Tensors (I understand Module’s forward() accepts Variables so no backward() needed )? I think I must be missing something but would like to know what is that, thanks.