Solved by albanD in post #2 
              
              
                Hi, 
Yes your description is correct. 
The main difference between 1 and 2 is that 2 is built with the nn package and handles nicely parameters and buffers. 
3 is useful in few cases: 

If an op is not yet implemented. But at the moment, all pytorch operations have a backward implemented (except one…