Uncomment the first pair of comments in the for loop. The gradient buffers have to be manually reset before fresh gradients are calculated.
Your for loop should be:
for t in range(500):
    y_pred = x.mm(w1).clamp(min = 0).mm(w2)
    
    loss = (y_pred - y).pow(2).sum()
    print(t, loss.data[0])
    
    w1.grad.data.zero_()
    w2.grad.data.zero_()
    
    loss.backward()
    #print w1.grad.data
    #print w2.grad.data
    w1.data -= learning_rate * w1.grad.data
    w2.data -= learning_rate * w2.grad.data