It updates other gradients, yes. As for second question, try doing x.requires_grad_(), but I doubt that checkpoint will be useful with such a big function, selective checkpointing of some resnet’s layers may do better.