I am trying to build a progressive autoencoder, for the encoder part I would like to prepend more layer on top of a ModuleList, how can I achieve that? I want to avoid copying and remaking my model every time I grow a layer.
eg:

# BEFORE GROWTH
self.layers = nn.ModuleList(  [ conv2d( 256, 512 ), nn.ReLu() ]  )

forward(self, x ):

    for layer in self.layers.children():
        x = layer( x )

 return x

** PREPEND module to the top of the encoder... **

# AFTER GROWTH
self.layers = nn.ModuleList(  [ conv2d( 128, 256 ), nn.ReLu(), conv2d( 256, 512 ), nn.ReLu() ]  )

forward(self, x ):

    for layer in self.layers.children():
        x = layer( x )

 return x

I have a working prototype model already but I was constantly destroying and making new nn.sequential which isn’t efficient at all.