Multi-GPU error
After I wrapped my model with DataParallel, this error happened: RuntimeError: Assertion `THCTensor_(checkGPU)(state, 5, input, gradOutput, gradWeight, sorted, indices)’ failed. Some of weight/gradient/input tensors are located on different GPUs. Please move them to a single one. at /home/soumith/local/builder/wheel/pytorch-src/torch/lib/THCUNN/generic/LookupTable.cu:17 My model includes an embedding() layer. Is this caused by embedding()? If so, any suggestion on how to do multi-gpu properly with embedding() layers inside the model? Thanks
Xia_Yandi
apaszke
peak
Not yet, the PR still needs some fixes.
apaszke
This is merged now. But you need to build from source if you need this change right away.
apaszke