I have worked on something similar. Not sure if this is right, @ptrblck can confirm if it makes sense.
I have three networks embed, classifier and adversary. The embed+classifier network take a gradient step with the adversary fixed and the adversary takes a gradient step with the embed+classifier fixed
In my training loop I did something like
embed.train()
classifier.train()
embedding = embed(input)
output = classifier(embedding)
classifier_loss = classifier_criterion(output, target)

adversary.eval()
with torch.no_grad():
  sensitive_output = adversary(embedding)
adv_loss = adv_criterion(sensitive_output, sensitive_target)

loss = classifier_loss + adv_loss

classifier_optimizer.zero_grad()
embed_optimizer.zero_grad()
loss.backward()
classifier_optimizer.step()
embed_optimizer.step()

and similarly for the adversary gradient step I got my classifier_loss with embed and classifier set to eval() and with torch.no_grad().