Solution has been posted:

Pytorch requires your to do  self.module_name = module  for things to work correctly. It’s okay to keep them in a list. Just do something like  setattr(self, 'emb_{}'.format(i), emb)  for each step in that loop.