Thank you for your reply. I did check the optimizer.state_dict()['state'] and it comes up as an empty dictionary for every call of the solver. As it turns out, the unexpected results are due to the data. I will continue experimenting with different UNet configurations, as the current model was just a very simple one, and will update the post if needed. Thank you for your suggestions and help @ptrblck!