That’s a convention used by DataParallel. It performs the split and copy to other device by himself. The user does not know how the data will be splitted, so you cannot copy the data to the other devices in advance.