In your first code snippet, the computational graph is created only once, so when you use for loops and try to backward on the graph for more than once without using retain_graph=True, the error will occur.
In you second code example, each time the for loop is executed, a fresh new graph is created and you can backward() through it once. If you try to backward in the loop for a second time, it will also fail:
import torch

x = torch.nn.Linear(3,3)
a = torch.autograd.Variable(torch.randn(2, 3))

for i in range(100):
    x_out = x(a)
    y = torch.sum(x_out)
    y.backward() # fine
    y.backward() # fail cause you are trying to backprop on the same net for a second time