If the encoders are trained separately why would you want the changes to one encoder to get reflected in another
If you want that to happen, you have to train all of them as part of the same model.
You can use ModuleList 1 and ParameterList 1 to hold the models and the embeddings respectively