You might want to uninstall the pytorch you already have installed?  Also clean the repo with python setup.py clean after that.
Also if the issue is still with test_api, you can try disabling building the tests with BUILD_TEST=0.