Because your model is set to train mode by default, pytorch is probably keeping the forward flow of the network with the specific inputs for future gradient computations. If you only want to use your model for inference, you should try calling model.eval() first, and also use with torch.no_grad(): before all calls that should not compute gradients.
If you want to train, however, you should try with an optimizer and loss function and follow the usual procedure, i.e:
model.train()
for i in range(2):
    # forward
    y = model(x)
    # reset previous gradients (free memory)
    optimizer.zero_grad()
    # compute gradients across the network
    loss.backward()
    # update weights according to gradients
    optimizer.step()