Hello all,
TLDR: I would like to update only some rows of an embedding matrix for words that are out of vocab and keep the pre-trained embeddings frozen for the rows/words that have pre-trained embeddings.
I’ve seen some solutions (e.g. here 116 which I got working) but from what I can see they mainly rely on maintaining another embedding matrix of the same size as the pre-trained/frozen one which is too slow in this instance for my use case (speed is crucial and this doubles the time per epoch in this case). What is the best (and fastest) way to do this?
I hope this is clear, please let me know if not (first time posting here).
Many thanks in advance,
Mark