My joy was premature. At the very first attempt to really use CUDA (which is ‘available’, according to PyTorch output), I am getting:
UserWarning: 
Found GPU0 Quadro K2100M which is of cuda capability 3.0.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability that we support is 3.5.

and:
RuntimeError: CUDA error: no kernel image is available for execution on the device.

Is there any possibility, after all, to use NVIDIA Quadro K2100M as CUDA enhancer for PyTorch?