justusschock:

2 trains the encoder together with decoder_SO and the other decoder will be trained separately.


If I want to train encoder using both information from decoder_SO and decoder_CO. So I need to use option 1, Is it right? I am not clear about your answer in second case