Depending on how complex my_func is, you may be able to get some mileage out of wrapping it in nn.DataParallel and running each of them on separate GPUs if you have several available. But you can also rewrite my_func to take a list of variables and manually batch certain suboperations together; that’s likely to be the fastest approach.