I followed this guide:

  
      
      GitHub
  
  
    

pytorch/pytorch 97

pytorch - Tensors and Dynamic neural networks in Python  with strong GPU acceleration


  
  
    
    
  
  


This hint is shown on the from page pytorch.org 45 e.g. when
[OS] OSX
[Package Manager] Source
[Python] 3.6
[CUDA] 9.1
The benefit is if you have conda installed there is no need to deal with all the X Tools stuff, you can simply use cmake from conda for compilation.
Citation from the guide mentioned:
export CMAKE_PREFIX_PATH=[anaconda root directory]
conda install numpy pyyaml setuptools cmake cffi typing

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch

# fix OS Version if nor on 10.13
MACOSX_DEPLOYMENT_TARGET=10.13 CC=clang CXX=clang++ python setup.py install