I will answer to this, because I stumbled upon this problem 3 years later:
You can fix this by writing a generic function, which creates an object variable for each ModuleList element.
Assume, you have all layers ordered(first called layer first) in self.all_layers of your network, which needs to use your nn.ModuleList(self.all_layers).
By using the function
def _make_object_variables(obj, layers):
    for i, layer in enumerate(layers):
        setattr(obj, f"layer_{i}", layer)

by just calling _make_object_variables(self, self.all_layers) at the end of your init function. This _make_object_variables just does the above mentioned: Creating a object variable for each layer in the format “layer_1”, “layer_2”, …, “layer_n”
Instead of iterating over your self.all_layers in your forward pass, you just can get the layer by using
        for i, _ in enumerate(self.all_layers):
            layer = getattr(self, f"layer_{i}")
            .......

This worked for me in a UNET structure very well to fix Dataparallel training.