I’m not familiar with your setup, but you could check how large each communication between the devices is and how much bandwidth your system allows (e.g. using NVLink could speed it up significantly etc.).