The official wheels statically link e.g. libcudart so that you can install the PyTorch wheels without a CUDA toolkit installation on your system (only the NVIDIA drivers need to be installed).
The build scripts can be found in pytorch/builder 19.