Thanks for the information regarding Caffe2!
Have a look at this issue 13 with some suggestions.
@ezyang suggests in this post 7:

If you want to use PyTorch master and Caffe2 together, please install them jointly with FULL_CAFFE2=1 python setup.py install , and make sure you don’t use any old pre-build binaries. I’m going to close this in favor of #10249 4 to track.