Hello @ElleryL,
Both cases you mentioned above are right.
But don’t forget dynamic update the output of the network when you increasing number of layers dynamically. I mean to say if you increasing layers into the current network by model.add_module(), it will not update the forward() process, so you should update the output like output = new_layer(model(input)) manually. Maybe there is a better solution.