As long as you keep track of gradients, I would say yes, but I’m no expert.
It’s hard to tell what is wrong without the knowing data, and a description what is the purpose of the model and side-functions.
I would suggest go with the debugger line by line with some toy data, and see manually if everything work as you’ve intended.
BTW there is no need to go with .to(device) for example in output = output.to(device), because the tensor should be already on the device, if the model1 is already on the device.