Ok, this gets stranger. I tried setting
torch.backends.cudnn.benchmark = True

and after running this code for the first time my PC just shut down immediately. Next time, provided benchmark mode is True, the code runs fine with the original data size.
It seems I’m getting several crashes like this, usually after I have restarted spyder’s kernel one or two times, using benchmark mode. However if I run the code on a fresh restart it seems to work. Sound like a low level issue.