You could add some timing information to those paths. For example, if all time is spent in parallel_apply you know there is something inside the model that’s causing this, instead of this custom nn.DataParallel wrapper. Alternatively, wait for the author to have time to debug this with you.