Which libgomp will be used depends on the compiler you used to build PyTorch. If you install gcc from anaconda, it will be linked with /anaconda3/lib/libgomp.so.1, if you use a system compiler, it will be linked with a system-wide lib dir. Both ways should work ok.