bigger issue is that Distribution objects are immutable, so you can’t push nn.Parameters inside them in __init__. Keep parameters in modules, create transient Distribution objects in forward, then saving & training will work correctly.