Hi,
Yes your description is correct.
The main difference between 1 and 2 is that 2 is built with the nn package and handles nicely parameters and buffers.
3 is useful in few cases:

If an op is not yet implemented. But at the moment, all pytorch operations have a backward implemented (except one or two linear algebra operrations) so that should not happen.
Your operation’s true gradients are not what you want. For example you could have an operation non-smooth gradients for which you want smooth gradients to help training.
Your operation would be slow and memory consuming to backward using auto-diff and so you implement the backward by hand to reduce memory usage (and potentially speed it up depending on the op). This is particularly true if your forward contains a very large number of small ops.