An easy way would be to replace them with nn.Idenity layers (assuming their input shape matches their output shape). Alternatively, you could also override the forward in a custom module and skip these layers directly.