I was running a simple network and found my new RTX 3080 is even slower than my old 2070 super. I checked the time used for each step. The backward propagation is really slow.
I am using pytorch 1.7.1 with CUDA 11.2.