Hi, I have a CNN model, which produces the index of an object in a vocabulary. The index is then embedded (by nn.Embedding) and fed into a language model for further processing. What I need here is end-to-end learning. However, the problem seems to be that the learning error cannot be propagated back to the CNN model through the embedding layer. Since the language model is pre-trained, I cannot make any change to the word embedding (i.e. nn.Embedding) in this case. Anyone has experience on that? Thanks a lot for the help in advance!