If you are dealing with use cases where not all parameters are used in the forward pass, find_unused_parameters might help and avoid running into hangs. However, I’m unsure if this would/should also work if the loss is not created, since this particular model wouldn’t be able to contribute to the current training step at all.