As you said, the Variable class tracks gradients.  The computation of the gradient has been defined by someone else for some ops, which is why you only need to write the forward method, as long as all your operations comprise only these particular ops.
You can see here that the backward method has been defined for some of the ops that you use: https://github.com/pytorch/pytorch/blob/master/torch/autograd/_functions/pointwise.py 183
So if you are going to define a custom operation that cannot be composed of these predefined ops, you will need to write your backward function.