Solved by ptrblck in post #2 
              
              
                I’m usually creating the criterion as a module in case I want to store some internal states, e.g. weight, a specific reduction etc. 
Also, I would say it basically depends on your coding style and the use case you are working with. 
E.g. if you are reusing the criterion in multiple places (e.g. GAN …