You would have to use nn.ModuleList instead of a Python list for self.roi_classifiers to properly register them as submodules.
Inside a module, all directly assigned nn.Modules and nn.Parameters will be registered.
Tensors, which should not be trained, can be registered as buffers via self.register_buffer.