So what happens is that DDP wraps the model, so the original model is ddp_model.model.
Then DDP has a forward method that essentially does the “distributed-admin” (getting the data to the various cards and collecting results and preparing for backward results to be treated).
Now you could run ddp_model.model.neg_log_likelihood, but that would run on a single instance, defeating the purpose of using DDP.
I think the easiest way to is probably to fold all entry points (that should be distributed) into the forward method with a flag “what” that you check for the various functions.
Best regards
Thomas