Thanks, this could be one way to solve that - by manually passing the gradients to backward. Using 2 optimisers in a simple GAN case is fine. However, for example if you have N modules in sequence and you want to only optimize a randomly selected subset of modules then creating optimizer for every new subset or manually passing the gradients (as you suggested) would be a pain. My use case is even more complicated.
I think that having such a no_backward_grad_accumulation context would be extremely useful, as it would provide a very simple (single line) way of excluding ANY subgraph from optimization. I am currently reading through the autograd code to find out whether its possible.