Having compared the results of the Autograd profiler between the two instances, the major bottleneck seems to be that there is a really long “Dropout” call (as in, there are two total, one takes 65 us and the other takes 24,228 us), and a really long “bernoulli_” call (there are two, one takes 15 us and the other take 24,049 us).
Also what seems to be apparent when I run the profiler is that even if I call .cuda() on my model, all the time is in CPU and CUDA is 0.00 for everything…
I do get the warning
Found GPU0 GeForce GT 750M which is of cuda capability 3.0.
    PyTorch no longer supports this GPU because it is too old.

however, everything runs and tensors show device='cuda:0'