Have you checked the relative cost of data transfer between CPU and GPU vs. the actual computation time of the model? I’m curious if the bulk of the time is spent on .cuda vs. the actual model computation on each GPU.