I also have a compilation error when trying to install from source:
Install the project...
-- Install configuration: "Release"
-- Installing: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTHNN.so.1
-- Installing: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTHNN.so
-- Up-to-date: /home/alexis/src/pytorch/torch/lib/tmp_install/include/THNN/THNN.h
-- Up-to-date: /home/alexis/src/pytorch/torch/lib/tmp_install/include/THNN/generic/THNN.h
-- Removing -DNDEBUG from compile flags
-- TH_LIBRARIES: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTH.so.1
-- Compiling with MAGMA support
-- MAGMA INCLUDE DIRECTORIES: /home/alexis/anaconda2/include
-- MAGMA LIBRARIES: /home/alexis/anaconda2/lib/libmagma.a
-- MAGMA V2 check: 1
-- Autodetected CUDA architecture(s): 5.0
-- got cuda version 8.0
-- Found CUDA with FP16 support, compiling with torch.CudaHalfTensor
-- CUDA_NVCC_FLAGS:  -DTH_INDEX_BASE=0 -I/home/alexis/src/pytorch/torch/lib/tmp_install/include   -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/TH -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/THC   -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/THPP;-gencode;arch=compute_50,code=sm_50;-DCUDA_HAS_FP16=1
-- THC_SO_VERSION: 1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/alexis/src/pytorch/torch/lib/build/THC
[  1%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorMath.cu.o
[  2%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o
[  3%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorRandom.cu.o
[  5%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorScatterGather.cu.o
In file included from /home/alexis/src/pytorch/torch/lib/THC/THCTensorMathMagma.cu:4:0:
/home/alexis/src/pytorch/torch/lib/THC/THCTensorMathMagma.cuh:5:19: fatal error: magma.h: No such file or directory
 #include <magma.h>
                   ^
compilation terminated.
CMake Error at THC_generated_THCTensorMathMagma.cu.o.cmake:207 (message):
  Error generating
  /home/alexis/src/pytorch/torch/lib/build/THC/CMakeFiles/THC.dir//./THC_generated_THCTensorMathMagma.cu.o


CMakeFiles/THC.dir/build.make:3869: recipe for target 'CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o' failed
make[2]: *** [CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....

before I run:
$ git clone https://github.com/pytorch/pytorch
$ export CMAKE_PREFIX_PATH=/home/alexis/anaconda2
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda80
$ pip install -r requirements.txt
$ python setup.py install

It seems that magma.h is not included, while I have the cmake message -- MAGMA INCLUDE DIRECTORIES: /home/alexis/anaconda2/include that looks ok…
Edit: Problem solved after updating my version of cuda to the release one.