At the end of day, we will need to use from ._functions import Scatter, if Scatter does not support list scattering, then we will not be able to scatter a list of tensors with non-uniform sizes. Do we really have to pad them to the same size? This will cause more memory and communication bandwidth comsumption. The ideal way may be scattering the list first into separate GPUs and then pad them or just do for-loop inside module.
I encouter this situation when I want to training a custom RNN on multiple GPUs.
Is there a way to Scatter a list of objects? I think technically we should be able to do it.