Hi, I have a ModuleList contaning many modules.
I use the modules in the ModuleList as modules composing the same layer, so in the forward function I iterate through the list and than concatenate the results of the single modules.
(Basically I simultate the old torch nn.Parallel).
Is there a way to perform it efficiently (in parallel) ? It is very costly when dealing with big ModulesList.