I don’t think you can rely on the hooks running deterministically. However, the list of device ids you passed into DataParallel (or the default [0, 1, 2, 3] assuming 4 GPUs) specifies the order in which your data gets split across devices.
For example, for device ids [0, 1, 2, 3], the first quarter of data will get sent to device 0, etc, etc.You can use this information to reconstruct the order of your outputs: each output captured by a hook will be on a specific GPU.