The best way to evaluate that is to try it  But here are my thoughts:
Since it’s been instantiated, and probably sent to GPU, it will consume some memory, as the parameters are created and must be kept somewhere. However, gradients won’t be computed, as it’s not present in the forward method. Also, if you send the whole model.parameters() to the optimizer, it might consume a bit more memory again? Not sure about this one.
In such cases, I would try to pass an additional parameter to the model, which defines if the layers have to be created or not, to save a bit of memory. The parameter won’t be kept if it’s just for initialization.