@osalpekar Yeah, I know and I don’t want to change device_ids.  I just want to know the flow of the program. I guess you mean len(self.device_ids) == 1 with 2 gpus. So when will the else condition be triggered ?