Thanks for the reply!  I do in fact need to accumulate a list after init.  ModuleList appears to only allow Modules and not Parameters.  I tried making a new Module that only has my Parameter in it, but once again the Parameter has a grad calculated but not applied by the optimizer.  Is the only way to do this by using premade nn.Modules?  I am just trying to do a simple dot product of my parameter vector and a layers output vector.  I could try to extend a nn.Linear to not be fully connected maybe?