Hi, I built my CUDA extension module following this 2 link. Everything works well when I only use 1 GPU. And its utilization is high (>90%). However, when I integrated it into my neural work and trained with 2 GPU, the utilization of each gpu is pretty low (≈50%).
Any ideas? thanks!
My environments:
Pytorch:1.1
CUDA:10.1
OS:Ubuntu 18
GPU: RTX2080ti