This in not a solution. I am just asking again. I am also facing the same problem. If I do first
loss.backward()
optimzer_for_one_part of the model.step()

some weights are changed and then for maximization of the another part:
(-loss).backward()
This time the again calculated gradient will not be the same because some weights of the same network has already been changed.
Ideal case would be, same gradient to maximize one part and minimize another part