Hello
(sorry, this is a duplicate of the post in beginner category. I don’t know where to post it)
I’m developing a machine translation model(Seq2seq), which have different source and target tokenizer. (As a beginner, I even do not know if this is the correct way to solve the machine translation problem.)
The problem happens when I want to divide shared(input, output) embedding into source/target embedding (I’m using T5ConditionalGeneration). After that, there is a function self.init_weights() and here they operate self.tie_weights() method.
As far as I know, the self.tie_weights() make output embeddings the same as input embeddings. But in my case, then target embedding will be changed and cause indexing error.
My question is:

Should I call self.tie_weights() in this case?
Is there any better approach that I can attempt?

Here is the related post below:
  
    
    
    What is the `tie_word_embeddings` option exactly doing? 🤗Transformers
  
  
    Hi, 
for some models there is this tie_word_embeddings parameter. I think it is for the text 2 text models. 
Can someone please explain what exactly this parameter is doing? 
Many thanks 
Philip
  


Thanks in advance for any answers!