Hi Thomas,
Thank you for your reply. I’m not sure I follow what you mean when you say just split out the rows as needed? I need some of the rows of the embedding matrix to have requires_grad = True and the rest set to False but it doesn’t seem possible to do this via slicing the embedding with a mask (which just denotes the out of vocab words) like:
self.embedding.weight[mask, :].requires_grad = True
Speed doubles as the model is very light so all the complexity/parameters resides in the embedding layers.
Apologies if this doesn’t make sense - I’m a little new around here so it’s likely my shortcoming.
Mark
P.S. The latter part of your reply was over my head I’m afraid.