One shortcoming I can think about is that you would need a random forwardpass to create the state_dict in order to pass it to the optimizer.
Also the same would apply it you save the trained state_dict, create the model in another script and try to load the state_dict.
Your model would be empty until you run a pseudo-forwardpass.
I don’t really like the idea. You could evade this by calling the forward in the __init__ method, but that would obviously make your approach useless, since you need to know the shape.