Do you think is because when I push the data in cuda() before inserting into the model, all the data is being pushed to the first GPU? If so, before feeding it into the model, how do I split the data into multiple GPUs?