Would it be possible to return the outputs in your forward method and calculate the loss on the default device?
This would be the vanilla use case, while it seems you’ve implemented backward as a class function?