You could initialize the second optimizer with only the FC2 parameters, i.e excluding main model parameters. This should mean that the 2nd optimizer won’t update the weights of B1.
You would still need to make sure that during training of auxiliary model, gradients are not stored in B1 during the backward pass, and I think if you turn off the gradients in one model you will turn them off in the second model as well if they share underlying memory.
Off the top of my head, a possible solution would be implementing some locking mechanism between the 2 models s.t when you train 1st model, requires_grad is True, and False for 2nd model. There might be easier solutions though.