i upgraded to torch 1.10.0 and torchvision 0.11.1 that have been built on a server using cuda 11.4.
now, i am getting a different error at the first call (probably during loading torch libs). the error does not point to a line in my code. but it seems related to loading distributed …
any idea what could be the cause of this error?
terminate called after throwing an instance of 'c10::Error'
  what():  Socket Timeout
Exception raised from recvBytes at /tmp-folder/pytorch_build_2021-11-09_14-57-01/moduleX/python3.7/pytorch/torch/csrc/distributed/c10d/Utils.hpp:619 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x55 (0x2b10898cb905 in VENV-NAME/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0xd6 (0x2b10898ad2a9 in VENV-NAME/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #2: c10d::TCPStore::doWait(c10::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::chrono::duration<long, std::ratio<1l, 1000l> >) + 0xe8 (0x2b10673a9058 in VENV-NAME/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)


the same versions installed using conda work fine on a local server.
thanks