Good point!
I just verified that it is handled in pytorch to still share the weights such that changing the embedding weight in one submodule (eg., net_a) affects the main embedding weight and other submodule (eg., net_b).

  
      gist.github.com
  
  
    https://gist.github.com/InnovArul/d66a9e2bf56507dd0803135fb2d910d4 145
pytorch_shared_module_weights.py
import torch
import torch.nn as nn


class SubModule(nn.Module):
    def __init__(self, embedding):
        super(SubModule, self).__init__()
        self.embedding = embedding
        self.fc = nn.Linear(200, 200)

This file has been truncated. show original