Thanks for replying. So I don’t see pybind11 in the include directory. I see ATen, c10, c10d, caffe2, fp16, TH, THC, THCUNN and torch. I do not have root rights on the server. Server has 20.04 Ubuntu. I installed torch via pip. I’m not sure why do we need to use apt for this or maybe you’re just showing it as an example if i understand correctly.
I want to train CTRL on a 24x4 GPUs but since it doesn’t have model parallel built in, it won’t even fit on one GPU with single batch size.