Thanks @ptrblck for pointing out that shortcoming. There might be some way to work around it by modifying load_state_dict or using placeholder values, but I suppose it will always be better to have __init__ actually initialize the module, not least because that’s what everyone expects.
Do you know of any commonly-used approaches to simplify (or make more readable) model-building code? I’m now thinking of something like a helper method which returns the output size of forward given an input size.