ptrblck, I met a troublesome bug.
Since the feature map of each layer is distributed on different GPUs when using DataParallel. How can I add (or other manipulation) them up? An error is got:
RuntimeError: arguments are located on different GPUs ?
I’ve searched this problem but couldn’t found a solution.