To be clear, I want the combined_loss to be used for performing a gradient step on the classifier+encoder networks.
This is how I currently do it (after taking into account the comments above)
encoder.train()
classifier.train()
z = encoder(x_batch)
y_hat = classifier(z)

adversary.eval()
for param in adversary.parameters():
    param.requires_grad = False
a_fixed = adversary(z)

opt_cls.zero_grad()
opt_en.zero_grad()

cls_loss = cls_criterion(y_hat, y_cls_batch)
adv_loss_fixed = adv_criterion(a_fixed, y_adv_batch, y_cls_batch)
cls_en_combinedLoss = cls_loss + adv_loss_fixed
# want this to work only wrt classifier and encoder
cls_en_combinedLoss.backward()
opt_cls.step()  # classifier step
opt_en.step()   # encoder step

The optimizer steps for classifier and encoder should be with respect to the combined_loss not just the cls_loss.