I can’t find the post anymore, but I read on these forums that it is better (or at least faster) to calculate loss on the CPU. I wonder whether that is true. What are the pros and cons of calculating loss on the GPU vs CPU. Particular concerns are speed and memory consumption. My assumption is also that if there is a notable difference, that this is magnified with larger batches.