Hi!
So I have a rather strange problem. I am interested in loading a large embedding file in my network, however when I use nn.Embedding my CUDA runs out of memory. So instead of loading the whole file, setting it to the weight parameter in nn.Embedding and setting require_grad to False, I pre-compute my input vectors and send them directly as embeddings.
I am able to do this because I have a server with larger RAM (but no GPU) at my disposal, so I am able to convert my input vectors to embeddings prior to training my model, save it as a pickle file and load it while training my model on GPU. I thought this would improve my performance, since I am now using a larger embeddings file.
However, it results in an almost 3% decrease in precision and 2% decrease in recall! I rechecked by following the same procedure but using the smaller embeddings file. This time there’s a 3% decrease in both precision and recall, with the earlier (larger) file giving marginally better results.
My question is:
Does torch.nn.Embedding make any changes to Tensors when require_grad is set to False? Or does it optimize the network in any way?
In other words, what would explain the decrease in performance when replacing nn.Embedding by feeding precomputed vectors?