alekhka:

My understanding was that by not including them in the forward pass they would automatically be dropped off from the network and wouldn’t be backpropped through. Is that true?


Basically you are right. The “missing” operations won’t be tracked by Autograd and thus the backward pass won’t be performed on them either.
However, these layers are still registered as parameters to the model, so they will show up in the print statement as well as in model.parameters().
Since the PyTorch is a dynamic framework, the computation graph will be re-created in each forward pass. So the next forward pass could indeed use the left out layers and e.g. train them in every second iteration.