If you have a sizeable network, I think taking the loop over the parameters as the outer loop is very likely worse than the other way round and taking the gradients of every parameter in one go even if you then need to disentangle them.
The other question is whether you really need the scores separately or are later doing something that would lend itself to rewriting as a single loss to take the gradient of.
Best regards
Thomas