To make sure the computation graphs are not detached at any point accidentally, you could print the gradients of the parameters for the current model, which should be updated after the backward call:
output = model_to_update(data)
loss = criterion(output, target)
loss.backward()

for name, param in model_to_update.named_parameters():
    print(name, param.grad.abs().max())

where you would have to use your different models for the dummy model_to_update variable.