Your pseudo code looks okay in general. But remember to call ‘supernet.requires_grad_(True)’ before you train the supernet cuz its grad was turned off in the first stage. Similarly, call subnet.requires_grad_(True) before you train the subnet.
Regarding your bullet points,

yes, it’s enough to call require_grad_(False) to disable this part’s update since the optimizer updates each parameter based on corresponding grad.
Yes, if loss is only based on subnetA then other nets’ gradient will be zero.
I’m not sure what you’re asking. Do you mean merging training of subnets and supernet?