Hi Andrea,
Thanks. You inspired me.
Though it will not work to just return a list of layers since the parameter that an optimizer take should be an iterator of tensors, you remind me that each layer is actually a Module so it has its own parameters method. And I found that a ModuleList also has a parameter method.
So I just wrote:
def get_other_parameters(self):
  return itertools.chain(self.linear_layers.parameters(), other.parameters(), ...)

And seemingly it works.
However, it’s still a little clumsy since I need to enumerate all the parameters I want to optimize. In cases where the number of parameters is huge while I only want to exclude several of them, it is neither efficient nor elegant.
Hence I still want to know if there is a better solution.