I just installed PyTorch on AWS g2.2xlarge machine with the default Ubuntu AWS image (ami-e13739f6). Here’s what I did:
sudo apt-get update
sudo apt-get install -y gcc make python3-pip linux-image-extra-`uname -r` linux-headers-`uname -r` linux-image-`uname -r`
wget http://us.download.nvidia.com/XFree86/Linux-x86_64/375.39/NVIDIA-Linux-x86_64-375.39.run
chmod 755 NVIDIA-Linux-x86_64-375.39.run
sudo ./NVIDIA-Linux-x86_64-375.39.run -a
sudo nvidia-smi -pm 1  # enable persistence mode for faster CUDA start-up

And then install NumPy and PyTorch
pip3 install numpy ipython
pip3 install https://download.pytorch.org/whl/cu75/torch-0.1.10.post2-cp35-cp35m-linux_x86_64.whl 
pip3 install torchvision

Now PyTorch works with CUDA
ipython3
>>> import torch
>>> torch.randn(5, 5).cuda()
 0.8154  0.9884 -0.7032  0.8225  0.5738
-1.0872  1.0991  0.5105 -1.2160  0.3384
-0.0405  0.2946  0.3753 -1.9461  0.0952
 1.6247 -0.8727 -0.6441 -0.8109  1.7622
 1.2141  1.3939 -1.2827 -0.3837 -0.0731
[torch.cuda.FloatTensor of size 5x5 (GPU 0)]

EDIT: updated instructions