If you are creating modules in the forward method, they were not properly registered in the model creation and thus never pushed to the device via model.cuda() or model.to().
Besides that, the parameters of these modules are also most likely not passed to the optimizer and recreated using random values in each iteration.
In your last code snippet you are creating the additional layers in the __init__, which is the right approach to register them properly.