Hi! I am working on a personal NLP project that involves word and character embeddings.
I want to merge together pre-computer word embeddings and word-embeddings built from character embeddings, through concatenation. The dimension of the pre-computed embeddings is [batch_size; sentence_length; 200], where batch_size is the number of sentences. The dimension for the char-based word embeddings is [number_of_words; word_length; 60], because when computing char embeddings I have to concatenate all words of all sentences in the batch to make the input 3D.
I also have, for char embeddings, the indexes that I can use to separate the words back into sentences. I reshape the output of the char embedding layer into [batch_size; sentence_length; 60] by doing this:
x_char = self.char_embeddings(x_char, x_char_mask)   # shape: [number_of_words; word_length; 60]
x_char = pad_sequence(np.split(x_char, splits[:-1]), batch_first=True)  # shape: [batch_size; sentence_length; 60]

In the snippet above, variable splits contains the position of the last token of each sentence. I am wondering if using np.split to slice the tensor will tamper with the computational graph built by autograd, blocking backprop to the char embedding LSTM. Can anybody help?